Can Rev as CGI pull an URL's HTML into a variable?

[Dave Cragg]

On 16 Aug 2006, at 20:18, Dave Cragg wrote:

>
> On 16 Aug 2006, at 18:54, RGould8 at aol.com wrote:
>
>> Hmmm, I'm afraid I'm having the same problem, using the script you  
>> provided.
>>  I do get 90% of the html, but the loading process just hangs.
>
> I'm seeing the same thing. Strange. Only about 16K of data comes  
> back instead of an expected 18K or so.

I may have figured it out. The following now works from here.
----------------------------------
#!revolution
on startup

start using stack "liburl_1.1.5_minus.rev"

put $QUERY_STRING into searchstring
put "http://www.imdb.com/find?s=tt&q=" & searchstring into searchURL
set the httpHeaders to "Connection: close" ##ADD THIS
put URL searchURL into databack
if the result <> empty then ##error occurred
    put "error" && the result into databack
end if
put "parameter   = " & urlDecode($QUERY_STRING) into tRet
put cr & "databack = " & databack after tRet
put "Content-Type: text/plain" & cr
put "Content-Length:" & length(tRet) & cr & cr
put tRet
wait 30 milliseconds ##helps with some servers
end startup
----------------------------------------
I noticed that the headers returned from the imdb server had a typo:

"Connneciton: close" instead of "Connection: close"

This would mean the Rev CGI wasn't closing the socket before it  
returned, which may have upset the Apache server. Adding the  
httpHeaders seems to have forced the imdb server to correct its  
response header.

But this probably suggests that we should always add a "Connection:  
close" header when making a url request from a CGI. I wonder if this  
is an engine bug, and that open sockets are not getting closed at  
shutdown.

Let me know if it works.

As for you curl users -- traitors. :-)

Cheers
Dave