Buffer size (was Looking for parser for Email (MIME))

[Alex Tweedly]

On 22/03/2016 14:24, Richard Gaskin wrote:
> Given that the engine is probably already doing pretty much the same 
> thing, would it make sense to consider a readBufferSize global 
> property which would govern the size of the buffer the engine uses 
> when executing "read...until <char>"?
>
> In my experiments I was surprised to find that larger buffers (>10MB) 
> were slower than "read...until <char>", but the sweet spot seemed to 
> be around 128k.  Presumably this has to do with the overhead of 
> allocating contiguous memory, and if you have any insights on that it 
> would be interesting to learn more.
>
Rather than a settable global property, it may be better to have a 
readable global property which suggests an optimal (or near optimal) 
size for reading.

Also, I'd point out that it is NOT "read ... until <char>", it is "read 
... until <string>" (according to the dictionary - haven't tried it yet).

This means that for reading MBOX format, you could do something like 
(untested)

put CR & "From " into tTerminator -- note the space at the end of the string

repeat forever -- !!
   read from file tFilePath until tTerminator
   if it is empty then exit repeat
   put it into tOneMailMessage
   -- and process that whatever way you want
end repeat

-- Alex.