CGI and find

[J. Landman Gay]

On 9/2/03 11:36 PM, Richard Gaskin wrote:

> While I've written apps years ago that supported HC-style find options, for
> myself I'll admit I've rarely had a need for them, or in all honesty it may
> simply be that I've been thinking too much in terms of whole-string serches
> and have overlooked some useful power there.  What sorts of situations have
> you found the "characters" or "words" options useful for?

All the "find" options are pretty useful. Suppose I am searching for 
"good dog":

find "good dog" -- finds any card which contains "good" and "dog" at the 
beginning of words, not necessarily together and not necessarily even in 
the same field. For example, will tag "goodly dogma" as a match but not 
"good underdog". Will tag "dog is good" as a match even though they are 
in a different order. Will tag a card if "dogma" is in one field and 
"good" is in another. This is hugely handy. For example, if I remember 
someone's first name and the city they live in, I can find them by 
seraching for "john san francisco" and even though the name and address 
are in different fields, the card will be found.

find whole "good dog" -- finds only the whole phrase, and only where the 
phrase is made up of whole words. I.e., won't find "good dogma". Similar 
to the "offset" solution, except that it won't tag matches if they 
aren't comprised of exactly the words in the search phrase.

find words "good dog" -- finds only whole words, but they can be in any 
order and not necessarily in the same field. Will tag a card as a match 
if one field contains "good eating" and another field contains "hot 
dog". Would not match if the second field contains "hot dogs" though, 
because "dogs" isn't the word we're looking for.

find string "goo do" -- I don't use this much because it takes too long. 
But it can be handy if you need to look for a specific sequence of 
characters, including the spaces and punctuation in the sequence. This 
one would not match "good dog", but would match "mr. magoo does it". The 
string does not have to be at the beginning of a word.

find chars "goo do" -- find each string of characters comprising each 
part of the search phrase. In other words, find any sequence of "goo" 
and any sequence of "do", not necessarily together and not necessarily 
at the beginning of words. Would tag "does magoo" as a match, even if 
the two words were in different fields. This variation is also 
time-consuming to run, but can be invaluable if you need to find partial 
strings that may be located in different fields anywhere on a card.

> While offset is indeed slower than mark for finding all occurances in fields
> across multiple cards, if the data were in a delimited list I'd be surprised
> if whole-string matches were not faster using offset (that is, provided a
> whole-string search is what you're after <g>).

In my cgi, I want to find a variety of matches, not just whole strings. 
But on the other hand, "find" wouldn't work at all if the delimited list 
isn't in a field somewhere. If a stack has only a single list to search, 
then putting that list into a variable and running an offset script on 
it would surely be faster. But if you have many lists scattered across 
many cards, "find" and its variations is a viable solution, and allows 
for a great deal of exacting control over what gets tagged.

Add to this the ability to set the dontsearch property of fields, cards, 
and groups, and you can really narrow down what gets searched and how 
fast it happens. If you don't want to search any of the fields in an 
entire background, just set its dontsearch to true and all the text in 
the background will be ignored. Handy.

> I have a subjective feeling of vague "muckiness" when using fields for raw
> data storage, with the object and style-run overhead....

I don't think the objects get in the way too much when the engine is 
running in non-GUI mode, but even in the IDE it works pretty well for 
what I'm doing. This particular stack is not very large though, only 
about 300 cards. There is no text styling anywhere, it's just a small 
database. The find/mark process in my tests was much faster than using 
offset to go through all the fields; a simple find/mark/process cards 
returns in 6 ticks. When I was using offset, the same search took 18-20 
ticks and I was more limited in what I could search for, since I could 
only get whole string matches.

This isn't to say that storing data in stacks is always a great idea. 
For larger data sets the engine breaks down and you need a real 
database. But for smaller data sets -- under 1000-2000 cards or so -- I 
have found stack-based storage to be easier to set up, easier to 
maintain, easier to script, quick to search and navigate, and supported 
by a large number of native commands in the language. I think it is an 
overlooked solution. I don't feel the need to put my little personal 
address book into an SQL database.

-- 
Jacqueline Landman Gay         |     jacque at hyperactivesw.com
HyperActive Software           |     http://www.hyperactivesw.com