Many Cards Versus One Card and a List Field

[Gregory Lypny]

Thank you for your responses Sarah, Mark, Paul, and Richard,

As always, thoughtful and insightful.  You've confirmed what I've  
learned with my experience with Revolution, comparing the one-card-per- 
record model with the one-card-as-a-display model, so now I'm  
convinced that I can abandon the former for most projects.  Richard, I  
would be interested in seeing some of those database handlers.  I've  
got a few bits and pieces and approaches of my own, many motivated by  
discussions on this list.  Here are three.

1.  I don't use the Filter command anymore because it is much too slow  
on big lists and is too cumbersome to script when the search is  
restricted to particular line items.  Instead, I use handlers with  
Repeat-for-each loops or Split to create arrays, probably like Paul  
does.

2.  For big data sets, I generally initialize a variable to be a  
number using arithmetic commands rather than "put" when it is to be  
used as an index or  iteratively in a calculation.  This means that  
I'll invoke the variable by writing "add 1 to x" rather than "put 1  
into x" to get things going, and subsequently rely on arithmetic  
commands.  This is motivated by an excellent thread on this topic by  
Wil Dijkstra in April, 2003.

3.  I've only begun to tinker with custom properties, and have found  
them to be handy for storing information on database structure (e.g.,  
first name is always the third item in any line of a Personal  
Information list field), but I keep the actual record data in a list  
field and back it up into plain text files.  I have found that scripts  
work much faster if custom props are pre-loaded into variables outside  
a repeat loop rather than being called anew at each iteration.  I've  
observed a ten-fold speed increase in some cases.

Regards,

Gregory Lypny

Associate Professor of Finance
John Molson School of Business
Concordia University
Montreal, Canada




 From Sarah:

On Sun, Jan 6, 2008, at 6:09 AM, use-revolution- 
request at lists.runrev.com wrote:

> I would recommend the list field-single card method for Rev. With
> Hypercard, marking cards by finding was super fast. With Rev this is
> not the case. I wrote a small database (about 4000 records) in
> Hypercard for some friends, using the one card per record method. When
> they changed to OS X, I tried just importing it into Rev but searching
> for multiple matches was unworkably slow. I converted it all to a
> single card plus data field and it was all good again. Plus the data
> is easier to back up and restore if I send them a program update.
>
> Cheers,
> Sarah

 From Mark:

> Sarah is right. Keeping data on individual cards makes Revolution
> extremely slow. A long time ago, I used the bible to create a
> database of approximately 32000 records for a test, i.e. 32000 cards.
> Running the test on a 350Mhz iMac, a search for a string could take
> an hour if this string was on one of the last cards. The same test in
> a HyperCard stack with 32000 cards took a few seconds at most.
>
> If you keep data in a custom property or in a file on disk, you can
> search strings about as fast as with HyperCard. Particularly if you
> have all data in memory and use an offset function or a filter
> command, you can perform searches very quickly.
>
> Best regards,
>
> Mark Schonewille

 From Paul:

> There are three big benefits from using the one-card approach with  
> Rev.
> The programs will run faster.
> The stacks will be smaller.
> It will be easier to build menus.
>
> BTW, I have found "repeat for each line whichLine" works faster than
> "lineOffset" and more reliable than "filter with/without".
>
> You will miss
> HC's ability to easily find data in a specific field.
> The ability to refer to data by field name, instead of by item  
> number in a
> record string.
> The ease of relayering fields in an entire stack by rearranging them  
> on a
> card.
>
> Good luck and best wishes.
> Paul Looney

 From Richard:

> While it can take a bit more work up front to set up, I think you'll
> come to love the flexibility of maintaining your own data storage.
>
> What HyperCard did was bind your data to the physical card record
> structure, which means you get one presentation (the detail view, or
> "card") quite easily but at the expense of any other presentation  
> (such
> as a list view).
>
> I've been storing most of the data my apps work with in tab-delimited
> custom properties for many years.  I have a lot of code which might be
> of use - I'll see about posting some of it in my Handy Handlers column
> at revJournal.com soon, but if you'll tell me a bit more about your
> project maybe I have something lying around which would be useful  
> right now.
>
> -- 
>  Richard Gaskin
>  Managing Editor, revJournal