text files with quotes

[David Vaughan]

The original problem was to take a record like:
1,"5/14/02, 10:00 AM",Pending,Batch0001,"This is a memo field, gee it's 
cool",45362
and to strip the quotes, while either removing the commas which were 
inside quotes but not the others (so the record can then be parsed by 
item) or alternatively to replace the commas outside quotes with tabs 
and leave those inside quotes alone.

At this stage we have four different but successful scripts (Mark, 
Geoff, Jan, myself) plus a practical suggestion here from Rob  to remove 
the problem at source. So, I ought to stop talking about this but last 
night as I dropped into bed I recognised a fifth solution which was 
still sufficiently elegant in the morning that I am publishing it anyway 
as grist for those who deal with similar problems. It exploits the fact 
that Rev perceives both a quoted string and a comma as single tokens.

   put field "data source" into inList
   repeat for each line inLine in inList
     put empty into outLine
     repeat for each token inTok in inLine
       if inTok is comma then
         put tab after outLine
       else
         put inTok after outLine
       end if
     end repeat
     put outLine & return after outList
   end repeat
   replace quote with empty in outList
   put outList into field "ready to parse"

Alas, runtime is off a little compared with our earlier solutions but at 
over 10k lines/sec it's simple and effective.

regards
David

On Friday, May 17, 2002, at 12:31 , Rob Cozens wrote:

>> This line is comma delimited, but if I try to separate it that way, the
>> interior quoted commas will mess things up,
>
> Hi Mark,
>
> Do you have any control over the format of the source files?
>
> If so, have the file prepared with a different item delimiter that 
> can't possibly appear in your data.
> --
> Rob Cozens
> CCW, Serendipity Software Company
> http://www.oenolog.com/who.htm
>
> "And I, which was two fooles, do so grow three;
> Who are a little wise, the best fooles bee."
>
> from "The Triple Foole" by John Donne (1572-1631)
> _______________________________________________
> use-revolution mailing list
> use-revolution at lists.runrev.com
> http://lists.runrev.com/mailman/listinfo/use-revolution
>