What is a token

[Bob Sneidar]

Hi all. 

Recently in an effort to help Mark Wieder with the formatting portion of GLX2 v4, I came across an issue where the block comments were not formatting correctly, and any handler with a block comment didn’t auto indent. 

In looking through the code, I discovered that the handler responsible for this was checking to see if the first TOKEN of a line was “/*” but when checking in the message box, “/*” is not a token! “?” is and “*” is, and any word is along with a list of special characters. 

I get that the dictionary states that this function should be used with trepidation, but my question is, since the 2 characters are like a reserved word, shouldn’t they be considered tokens, along with the reverse “*/“? Note that block comments do not have to be at the beginning of a line. 

So the question is, should I submit this as a bug, or no?

Bob S