[OT] More on false timeouts and headers

[J. Landman Gay]

On 5/20/2015 6:47 PM, Dave Cragg wrote:
>
> It should just return the content without the chunk marker thingies.
>
> I’m using an older version. But if you run this script in a card with two fields, you should see it is using "chunked" encoding. The returned data in field 2 has no chunk markers. Is that what you see with your version?
>
> on mouseUp
>     libURLSetLogField "field 1"
>     put "http://www.lacscentre.com/chunkcheck/chunkcheck.php" into tUrl
>     put url tUrl into field 2
> end mouseUp
>
> I don’t know of any way to force the remote server to not use chunked encoding.
>
> Can you see the pattern of the chunk markers in the data you are getting back? The markers should be on a separate line starting with a hexadecimal value indicating the chunk length, followed possibly by some semicolon delimited parameters, and then a CRLF line ending.

Thanks for this, Dave. I don't see any chunk markers in the file with 
the version of libURL I'm using, I see rows of plain text. But even 
though the Content-Encoding header indicates it's a gzip file, it isn't 
coming in as one. The data I'm getting from my client's server is a 
compressed gz file and uses a "Content-Transfer-Encoding: binary" 
header. It looks like the server sends plain text in your test file.

Last week I wrote a little utility to look at the first few bytes of the 
files as they arrive. Here are the first 10 bytes (in decimal) of a file 
that fails:

   49 57 13 10 31 139 8 0 225 192

And here are the first 10 bytes of a file that gunzips okay and is working:

   31 139 8 0 48 225 75 85 0 3	

This is the raw data that is returned to my script after a "get url" 
command. It does appear that the first four bytes of the "bad" file 
include the chunk markers. The "31 139" sequence are the markers for a 
gz file and that sequence is what my script is looking for. In the "bad" 
file it doesn't show up until position 5.

Have you tried a gz file sent with "content-transfer-encoding: binary"? 
We did try sending the gzip as plain text instead of binary and it 
worked okay, but with the changes LC is making, libURL will try to treat 
text as unicode in LC 7. So we figured setting the transfer-encoding to 
text was risky (even though it worked in LC 6.)

-- 
Jacqueline Landman Gay         |     jacque at hyperactivesw.com
HyperActive Software           |     http://www.hyperactivesw.com