[OT] More on false timeouts and headers

[Dave Cragg]

> On 21 May 2015, at 20:07, J. Landman Gay <jacque at hyperactivesw.com> wrote:
>> 
>> I think that will pick up any Content-Transfer-Encoding header if it
>> exists before a Transfer-Encoding header, and therefore miss the
>> "chunked" value, and so not try to de-chunk the data. (I guess that
>> might be considered a bug in libUrl. Even if
>> Content-Transfer-Encoding is not valid, something like
>> X-Special-Transfer-Encoding is valid, and would probably trip up
>> libUrl)
> 
> That's exactly what's happening, you've nailed it. I did a quick check yesterday and found over 1900 insertions of the chunk markers inside a 550K file, so it's being sent to my script as a raw data stream. We still don't know what's inserting some of the header lines, it isn't anything our own scripts are doing. Something from AWS maybe.
> 
> So is this something you think I should report? Or is it just a side-effect of working with different servers? There is a workaround; specifying a Content-Length header appears to eliminate both the "Content-Transfer-Encoding" and "Transfer-Encoding: chunked" headers.


You could modify libUrl. :-) Perhaps not an option if you have many users of the software. But it might be worth trying to confirm this is the problem:


--------- at your own risk--------
This is for the version of libUrl that comes with 6.6.3 and 7.0.3

- Message box: edit script of button "revLibUrl" of stack "revLibrary"

- find the handler "on ulParseHeaders x" (about line 930)

- near the beginning locate this line:
     set the lastRhHeaders of me to laRhHeader[laUrl[x]] ##set property

- Immediately after that line, insert the following:
   
   filter lines of laRhHeader[laUrl[x]]  without  "*Content-Transfer-Encoding:*"

——————————————————

This is a bit of a hack, and a better way of parsing the headers is probably needed.


> There is a workaround; specifying a Content-Length header appears to eliminate both the "Content-Transfer-Encoding" and "Transfer-Encoding: chunked" headers.

I don’t quite follow. Are you in a position to set a Content-Length header on the response? 


Mark wrote:

> And RFC1341 specifically states
> 
> "Unlike Content-Types, a proliferation of Content-Transfer- Encoding values is 
> undesirable and unnecessary."


Under rfc-talk, that sounds a little like "shouldn’t contain" rather than "mustn’t contain". I can see blame going in both directions here.

Dave