Best modality for rather large stacks

[Alex Tweedly]

Gordon Tillman wrote:

>
> On Dec 16, 2005, at 13:18, FMoyer at aol.com wrote:
>
>>
>> Can anyone advise on the best mode? At the moment I am inclined to  
>> create a
>> one-card stack that links to a text file. In the text file, one  line 
>> of text
>> would contain all of the data for one card. So to "go" to card 533  
>> the computer
>> would simply read line 533 of the text file, parse it, and set up  
>> the data
>> onto the fields.   This method seems really good as searches are  
>> very fast using
>> "lineoffset", and if things get fouled up I can simply open the  text 
>> file in
>> any word-processor and fix it. I also like the fact that I don't  
>> have to
>> continually save the stack -- in my script the text file would be  
>> continually
>> updated as fields are changed, cards created/deleted.
>
>
>
> I would recommend that you use the altSQLite external from http:// 
> www.altuit.com/webs/hemingway/AltuitCover/default.htm.
>
> This external is a wrapper around the excellent SQLite database  
> engine.  You can read more about SQLite here:
>
> http://sqlite.org/
>
> Works very well, is extremely fast, and can handle even much larger  
> databases that what you are needing.  Plus it is less expensive than  
> than the Valentina route.  The database file is also fully cross- 
> platform.
>
sqlite sounds like a good option - but I'm not sure it is needed. For 
5000 records of 50 fields, that's not a scale of problem that *requires* 
any database. There may be other aspects of your problem that cause you 
to need a database (complex searches or sorting, maybe), but for many 
purposes, the simple text file with CR + TAB delimiters should be adequate.

I have a couple of such projects - the larger one is 8-9,000 records of 
40 fields per record - around 1.4 Mb total data. Straightforward 
searches - e.g. filtering to display only those records that match a 
couple of values or even regexes in specific fields - are, to all 
intents and purposes, instantaneous. I don't do much in the way of 
inter-record combinations, though I do some (it's a pedigree database, 
and I do things like multi-generation pedigrees and offspring reports). 
This is done with a simple text file for storage, with a the data held 
in a Transcript variable.

You should be careful about how / when you save changes (I do so only on 
user command, and keep N generations of backups, where N is 
user-selectable) - but my database is close to read-only - changes are 
infrequent and usually arrive in a large batch.

My advice is to not leap into using a more complex solution until you've 
proven a need for it.

-- 
Alex Tweedly       http://www.tweedly.net



-- 
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.371 / Virus Database: 267.14.1/204 - Release Date: 15/12/2005