Chunk order must be small to large

[Richard Gaskin]

Jim Bufalini wrote:
...
> And this is logical as, a. You can set itemDelimiter and lineDelimiter but
> not the word delimiter (smallest chunk, which word boundary has undergone a
> change recently in Rev 3.0), b. When parsing text, words are generally
> smaller chunks than phrases, which are usually delimited with commas, c. As
> has been pointed out, you can override this parsing order using parenthesis,
> and lastly d. All that's needed here is *item 1 of "aaa,bbb"* ;-)

And there's also e. you get a lot of raw speed by letting the parser 
safely make logical assumptions about chunk order.

I asked Dr. Raney about this once, and he said that the other 
interpreters waste a lot of clock cycles letting you shoot yourself in 
the foot.  He trimmed a number of syntax elements down, often removing 
support for non-intuitive or rare uses (like the mystery world in which 
a word is longer than a line) so that he could deliver blazing 
performance in return.

In addition to enforcing a logical order to chunk sizes, other examples 
of this sort of optimization include:


- You cannot use reserved tokens as variable names.

   Writing "put word 1 into word" makes code hard to read.  So hard,
   in his opinion that it wasn't worth the parsing cost.


- You cannot override built-in functions and commands.

   It may be tempting to want to implement some custom behavior for,
   say, the truncate function.  Raney felt that all that did is risk
   confusing anyone who calls that function while your handler is in
   the message path, so in his view it didn't merely expode the size
   of the token table, it polluted it.


I went to the mat with him over this one years ago when I was porting an 
HC project.  He said, "If you need a custom behavior, use a custom 
name."  I said, "But I *need* to override the built-in function!" and he 
asked simply, "'Need'? Why?"  I couldn't think of a truly necessary case.

He said if I could ever find a true need for this he'd reconsider.  I 
told most everyone I knew; we put a lot of time into trying to think of 
a case.  No luck.  In the end we came to appreciate his edict: "If you 
need a custom behavior use a custom name."  Keeps code readable, lets 
folks know what to expect when they call something.


For those coming into Rev fresh from HC these things go against 
long-standing habits, and will appear strange or even "wrong" at first.

And then you start really working with it, learning these differences 
and moving on, and the more you code with this new engine the more you 
find yourself saying, "Man, this thing is fast!".

It's no accident.

It's a very carefully pruned token table.

--
  Richard Gaskin
  Fourth World
  Revolution training and consulting: http://www.fourthworld.com
  Webzine for Rev developers: http://www.revjournal.com