Fastest memory based search technique (was: Adding 1 to an array)

[Jim Ault]

I would recommend that you try to institute a controlled word dictionary,
rather than let the user create key words since this will defeat the
description of 'instant'.  Even with very fast databases, the "SQL join"
operation (which sounds like your method) can get slower and slower.

You might want to send a note directly to Eric Chatonet about this because I
believe he has done massive work and study on this topic (and multi-lingual
to boot!).  The key word for searching this archive I believe would be
"stemmer",  which is a way of defining which words are similar.  This would
be overkill for your project, but the fast techniques he uses could be
applied in some form.

Certainly you are developing a specialized application, but it is a road
well-traveled by many.

Hope this helps.  I have only done rudimentary indexing and on very small
data sets.  The "filter var with .." and "filter var without .." are speed
champs with short line-length lists.

Don't forget the idea adding a comment line at the beginning of the handler
that contains the keywords used in a single location.

on doThisForMe
--tagg doThisForMe calcPixels repeat for each union using

end 

then harvest them by...
filter handlersText with "--tagg*"  =>

Jim Ault
Las Vegas


On 6/24/07 4:32 AM, "David Bovill" <david at openpartnership.net> wrote:

> Thanks Ken.
> 
> On 24/06/07, Ken Ray <kray at sonsothunder.com> wrote:
> 
>> on mouseUp
>>   put 1 into tA[1]
>>   put 2 into tA[2]
>>   put 3 into tA[3]
>>   put tA + 1 into tA
>>   put tA[1],tA[2],tA[3]
>> end mouseUp
> 
> 
> On 24/06/07, Jim Ault <JimAultWins at yahoo.com> wrote:
> 
>> Does the list of tags change or is it static?
>> 
> 
> Its a general technique for keyword indexes I'm looking at, in this example
> it is for Rev handlers, each of which can have many keywords. It's the "find
> similar" algorithm I posted about earlier - so if a handler has say these 6
> keywords - which other handlers have similar keywords. Or another way of
> putting it if a user types in some keywords, how can we find related
> handlers in the library?
> 
> These are the types of technique I'd like to compare:
> 
>    1. My very old stack searched for keywords in a field, and i am
>    looking for a better way
>    2. Use indexes and loop though them with repeat
>    3. Use arrays and use union and intersect (current experiment)
>    4. Use database
> 
> I'm interested to see how the old style card index compares for speed with
> the rest.
> 
>  This is what i am thinking;
> 
>    1. Users will type in (or select from a list) the most important
>    keywords first
>    2. Keywords can be in any order
>    3. Cannot index all keyword combinations - so need fast dynamic lookup
>    4. indexes of keyword => handlerNames are useful so lets store them
>    somehow
> 
> If from each keyword i can fetch an index of handler names from an array
> then I have several indexes - then i guess i just need the fastest way to
> calculate unions of indexes. I'm thinking that arrays and union will be
> faster than repeating through and index or array.
> _______________________________________________
> use-revolution mailing list
> use-revolution at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-revolution