[OT] More on false timeouts and headers

[Dave Cragg]

> On 21 May 2015, at 04:17, J. Landman Gay <jacque at hyperactivesw.com> wrote:
> 
> Thanks for this, Dave. I don't see any chunk markers in the file with the version of libURL I'm using, I see rows of plain text. But even though the Content-Encoding header indicates it's a gzip file, it isn't coming in as one. The data I'm getting from my client's server is a compressed gz file and uses a "Content-Transfer-Encoding: binary" header. It looks like the server sends plain text in your test file.

Jacque,

Is there really a "Content-Transfer-Encoding" line among the headers? If so, that may be part of the problem. As far as I know, "Content-Transfer-Encoding" is not a valid http header. (It’s used in email transfers I think.) libUrl parses the headers looking for certain header types. It contains the line:

 put lineOffset("Transfer-Encoding:",laRhHeader[laUrl[x]]) into tCodeLine

I think that will pick up any Content-Transfer-Encoding header if it exists before a Transfer-Encoding header, and therefore miss the "chunked" value, and so not try to de-chunk the data. (I guess that might be considered a bug in libUrl. Even if Content-Transfer-Encoding is not valid, something like X-Special-Transfer-Encoding is valid, and would probably trip up libUrl)

Out of interest, I modified the script from the previous mail to this:

on mouseUp
    set the httpheaders to "Accept-Encoding: gzip, deflate"
   libURLSetLogField "field 1"
   put "http://www.lacscentre.com/chunkcheck/chunkcheck.php" into tUrl
   put url tUrl into field 2
end mouseUp

The server now returns gzipped data, but libUrl still strips out the chunk markers.

> 
> Last week I wrote a little utility to look at the first few bytes of the files as they arrive. Here are the first 10 bytes (in decimal) of a file that fails:
> 
>  49 57 13 10 31 139 8 0 225 192

Those first four bytes are the chunk marker line.