Cards vs Listfield/custom properties revisited

[J. Landman Gay]

Lars Brehmer wrote:
> I followed the discussion about using stacks of many cards vs. a 
> listfield vs. a custom property vs. a database.
> 
> I use a data stack with around 5,000 cards with anywhere from 4 or 5 to 
> over a hundred custom properties in each (out of 250 possible), so I 
> decided to try everyone's advice.  I created a custom property with 1 
> line with up to 250 items for each of the data cards.
> 
> My first speed tests were quite promising at first (example: what took 
> 1765 milliseconds the card way took just 60 milliseconds the new way.) 
> Being a non coder type (please be patient with my lack of expertice), I 
> haven't been able to find ways to translate all of the search and sort 
> operations I do. Most things rely on the very basic
> 
> mark cards where some custom property = some condition
> lock screen
> lock messages --the data stack has an openCard handler that affects my 
> main stack
> repeat for q = 1 to the num of marked cards
> go next marked card
> do something
> end repeat
> unmark all cards
> --mark cards the way they were before this operation
> unlock messages
> unlock screen

The bottleneck here is the "go" statement. Even with locked messages and 
a locked screen, it takes a long time to go to a card. In most cases, a 
repeat loop can be changed to use remote references to cards, which are 
much faster:

repeat with q = 1 to the num of marked cards
  do something to fld "myfield" of cd q -- remote reference
end repeat

Just this one change will increase the speed of the repeat loop 
tremendously. This doesn't address your question about dropping the 
card-based storage in favor of variable-based storage though.

> 
> While this has always been fast enough for me, I am trying to learn 
> better ways to do things, but after trying such things as filter with or 
> match etc, I have found no way do something that has the same result as 
> "mark cards where ...". Is there anything that would have the effect of 
> the (apparently not possible) command "mark lines in it where ..."? I 
> suppose it would look something like this:
> 
> put the mData of this stack into gData --loads that big custom prop
> somehow get rid of lines where something is not in a certain item of 
> those lines
> set the tempData of this stack to gData
> do things with the tempData of this stack

I wrote a school library system that uses the variable-based approach 
that addresses these things. For the most part, sorting is very easy:

   sort lines of tData by item <whatever> of each

Finding things takes a little more effort, but can duplicate the marked 
card approach. The "repeat for each" structure is very fast and can be 
used effectively to find data in the variable in many different ways. 
For example, to find all instances of "dog" in the variable:

repeat for each line L in tData
  if tData contains "dog" then put L & cr after tOutput
end repeat

The variable "tOutput" now contains only those lines that have the 
string "dog" embedded anywhere. If you only want to find whole items 
that equal "dog":

repeat for each line L in tData
  if "dog" is among the items of L then put L & cr after tOutput
end repeat

You can do combined searches too; for example, if you want to find 
either "dog" or "cat" as whole items:

repeat for each line L in tData
  if "dog" is among the items of L or "cat" is among the items of L
  then put L & cr after tOutput
end repeat

Or only those lines containing both "dog" and "cat" as individual items:

repeat for each line L in tData
  if "dog" is among the items of L and "cat" is among the items of L
  then put L & cr after tOutput
end repeat

There are endless variations to this. You can find all lines that 
contain one thing but specifically do not contain another:

repeat for each line L in tData
  if "dog" is among the items of L and "cat" is not among the items of L
  then put L & cr after tOutput
end repeat

And so forth.

-- 
Jacqueline Landman Gay         |     jacque at hyperactivesw.com
HyperActive Software           |     http://www.hyperactivesw.com