[Gambas-user] using a "file system database"
Kevin Fishburne
kevinfishburne at ...1887...
Fri Apr 15 09:22:37 CEST 2011
I'm in the early phases of creating a "database" that uses the file
system for data organization rather than a traditional software database
such as MySQL, etc. I'm hoping that this could be faster since my
requirements are very specific and (I think) don't need a general
purpose database.
I will have 4,194,304 "cells", each of which has about three datafiles
that will need to be opened, read from, written to and closed regularly.
I could consolidate the three files into one, reducing the number of
files, but this would increase the amount of time taken to parse the
files and slow the program significantly.
I'm considering dividing them into hierarchies of directories to avoid
having four to 16 million data files in the same directory. Initial
tests hit file system (or file space, not sure yet) limits.
Does anyone have any insights into the ext3/4 filesystem, possible
alternate file systems, and databases to know what the best solution
would be for this type of problem? Any insights into the most efficient
way to create hierarchies of directories and files would be appreciated
as well.
--
Kevin Fishburne
Eight Virtues
www: http://sales.eightvirtues.com
e-mail: sales at ...1887...
phone: (770) 853-6271
More information about the User
mailing list