Bug with token chunk type?

[Dave Cragg]

On 20 Mar 2007, at 14:50, Richard Gaskin wrote:

>
>> From the docs, "The token chunk is jmplemented for the Transcript
>> language, and probably isn't suitable for use in a general-purpose
>> language parser."
>> I've always taken this to mean that "token" is the engine's  
>> private  property, and we have no basis for questionning how it  
>> works. We can  guess, but we can also expect to be surprised.
>
> When I'm delivering a product I try to avoid guessing. :)

Me too. Which is why I avoid using "token" in my scripts. :-) But as  
usual, I suspect we agree on this. My point was that the docs state  
that "token" isn't suitable for general-purpose language parsing.  
That sounds to me like "use at your own risk".

Cheers
Dave