[OT] More on false timeouts and headers

[Mark Waddingham]

On 2015-05-21 23:44, J. Landman Gay wrote:
> To be honest, you've been more help than anyone else, for which I'm
> really grateful. She originally contacted the Passenger forum and they
> had little to offer outside of "your app should be handling the
> chunking. Not our problem." And then they locked the thread. So we
> tinkered around with dozens of things for days without any luck.
> 

In a way I think she is correct. Looking through the RFCs nowhere does 
it say that 'Content-Transfer-Encoding' must not appear in HTTP headers, 
all it says is that 'HTTP does not use Content-Transfer-Encoding' - the 
implication being that it should be ignored and not processed.

Indeed, it does suggest in section 19.4.5 of RFC2616 (HTTP 1.1 spec) 
that you might see Content-Transfer-Encoding headers in HTTP - they are 
there so that the body can be 'passed through to MIME compliant 
processes' which require the Content-Transfer-Encoding. (HTTP, from what 
I can see assumes and requires that Content-Transfer-Encoding be 
binary). My interpretation is this: you *may* find 
Content-Transfer-Encoding headers in HTTP as that means down-stream 
processors of the data being transferred (which are MIME compliant, 
rather than HTTP compliant) will work with it; however, HTTP is only 
required to process binary CTE thus if something other than that is 
specified it could be considered an error in the sender of the data.

Reading between the lines, I suspect Passenger cleaned up / improved 
their data -> MIME / HTTP encoding process unifying it and as a result 
HTTP headers now contain a Content-Transfer-Encoding when they did not 
before. (Strictly speaking I think that if Passenger were sending 
Content-Transfer-Encoding headers which specified something other than 
binary then it would be in error - but I suspect it will be sending ones 
specifying binary so it is - at least with a suitable interpretation of 
the RFCs - not in error).

Thus it seems to me that Dave is correct - this is actually a bug in 
libURL - it is being too lax when parsing headers and interpreting a 
header it should not be thus meaning it is not entering into 'chunked' 
mode even though it is explicitly specified. This is, of course, 
assuming the Content-Transfer-Encoding header appears before the 
Transfer-Encoding header in the full HTTP message body.

So, if indeed the headers being returned from the server are:
    ...
    Content-Transfer-Encoding: binary
    ...
    Transfer-Encoding: chunked

Then libUrl should be ignoring the CTE header, and carrying on its merry 
way - in this context Dave's patch to libURL should solve the issue for 
you.

Mark

P.S. Having read a little more of the RFCs - I *think* it is only the 
quoted-printable / base64 CTE's which are explicitly not allowed in HTTP 
- 7bit/8bit/binary are all equivalent when being passed through HTTP. 
The upshot is that the CTE being specified in an HTTP context is 
'allowed' as a hint to a future MIME-compliant processor about the 
encoding of the message.

-- 
Mark Waddingham ~ mark at livecode.com ~ http://www.livecode.com/
LiveCode: Everyone can create apps