Words Indexing strategies

[Bernard Devlin]

On Wed, Feb 10, 2010 at 2:56 PM, Alejandro Tejada
<capellan2000 at gmail.com> wrote:
> No, it's not a search inside the displayed article.
> It's a global search, within a general index created
> using all words from all articles of Wikipedia.
> (I do not believe that it's necessary to load this full
> index in memory, instead just open specific parts
> of this index when users start searching)

OK, so that's why you mention the different files for each letter of
the alphabet.

I'm still a bit confused.  Normally an index would indicate a location
for an indexed term  That's what I assume your general index files are
doing.  What are the key terms like in this index, and what do they
point to?  Can you give us some examples?

> For this reason, i am looking for advice to create an
> index structure that allows to implement a fast search
> algorithm, using multiple words (and boolean logic, if
> possible), similar to Wikipedia's own search engine or
> (better yet) just like google. :-)

To my confused and befuddled mind, it sounds like you are wanting to
create an index of the index.  That can't be right :-)

I'm no expert in search algorithms.  I have been hoping someone else
would jump in who has done this kind of thing before.

Are you wanting a pure, rev-only solution i.e. are you doing this to
demonstrate what can be done using Rev alone?

Bernard