What is a token

[Mark Waddingham]

On 2020-04-11 22:22, Bob Sneidar via use-livecode wrote:
> So the question is, should I submit this as a bug, or no?

No you shouldn't as it isn't a bug :)

The 'token' chunk is a wrapper around the internal routine the engine 
uses to divide a script text up into tokens which are then parsed. This 
routine also does the 'skipping' over of whitespace and comments. As 
'/*' is part of a comment it is ignored, just like any spaces in between 
actual tokens.

For example:

    put token 2 of "/* this is a comment */ put x into y"

Will put 'x' because the only 'tokens' in that string are 'put', 'x', 
'into' and 'y'.

If you want to see if a line starts with a multi-line comment then you 
should do:

   if word 1 of tLine begins with "/*" then ...

Warmest Regards,

Mark.

-- 
Mark Waddingham ~ mark at livecode.com ~ http://www.livecode.com/
LiveCode: Everyone can create apps