definition of "word"

[Peter M. Brigham]

In playing around with this I came across the following problem.

if tSentence =
   "I think animal testing is a terrible idea: they get all nervous and give the wrong answers."
(including the quotes),
then word -1 of tSentence =
   "I think animal testing is a terrible idea: they get all nervous and give the wrong answers."
as we all know.
And word -1 of
   I "think animal testing is a terrible idea: they get all nervous and give the wrong answers."
is
   "think animal testing is a terrible idea: they get all nervous and give the wrong answers."

If tSentence =
   ("I think animal testing is a terrible idea: they get all nervous and give the wrong answers.")
then word -1 of tSentence =
   ("I think animal testing is a terrible idea: they get all nervous and give the wrong answers.")
Hmmm. OK, a "word" includes immediately adjacent punctuation.

But if tSentence =
   (I "think animal testing is a terrible idea: they get all nervous and give the wrong answers.")
then word -1 of tSentence =
   )

I've complained before about the treatment of a phrase in quotes as a single word, but the minimum requirement for a computer grammar is consistency.

I'm looking forward to the proposed new implementation of this token.

-- Peter

Peter M. Brigham
pmbrig at gmail.com
http://home.comcast.net/~pmbrig