New chunks

[Peter M. Brigham]

On Mar 13, 2014, at 11:51 AM, Jim Hurley wrote:

> I have a poor man’s version that is workable.
> 
> However just ran across a sentence in today’s NYT that I will have to include:
> 
>        (“What they’re offering people is a full stomach and an empty soul.”)
> 
> I had dealt with the quote beyond the period and the paren beyond the period, but not both.

Here's a simple function I've been using. The new LC grammar revisions will make it obsolete eventually.

function naturalWord tWord
   -- strips punctuation from HC-style "words" fore and aft
   -- to return something closer to what is normally understood as a word
   put "abcdefghijklmnopqrstuvwxyz1234567890" into tAlphabet
   -- numerals included to cope with numbers and things like "HTML5"
   repeat while char 1 of tWord is not in tAlphabet
      delete char 1 of tWord
   end repeat
   repeat while char -1 of tWord is not in tAlphabet
      delete char -1 of tWord
   end repeat
   return tWord
end naturalWord

It's simple, and therefore somewhat simple-minded -- it will miss some cases, but I've found it useful.

"Every complicated problem has a simple, easy, obvious, wrong answer."
                -- H. L. Mencken

-- Peter

Peter M. Brigham
pmbrig at gmail.com
http://home.comcast.net/~pmbrig