CGI POST Failure to read Stdin
Pierre Sahores
psahores at easynet.fr
Thu Apr 8 09:01:44 EDT 2004
Hi Katir,
I experimented this,a long time ago... At least with the 2.3.2 and
above issues of the engine, MC/Rev is failing 1 time peer 20 periods in
handling "POST" requests in CGI mode.
It's a sad reproductible bug we spoken about with Scott Raney, a long
time ago, on and off-list, without getting any way to solve
definitivelly the problem.
Linux, MacOS X, and Apache on both platforms are OK. The problem is,
specifically, a bug inside the MC/Rev engine.
The only workaround we can use at this point is to avoid to use MC/Rev
server-sided stuffs in CGI mode. For my own, i choosed to run all my
MC/Rev server-sided apps as sockets-listening apps, not only because
the CGI mode bug but even because it.
The way is to stick a non-MC/Rev sockets listener behind Apache (php,
for my own but, any scripting langage that can be handled as an Apache
module - speed considerations - is probably OK too, alike perl or
python) and to let it translate the Apache default entry port (:80) to
the one the MC/Rev app is waiting for messages on ( < 1024, for a root
protection is best but it's up to you want).
This way is very easy to manage and works like a charm : no more
failaitures at all and when i say that, it meens that 0% of my "POST"
requests are failing. If it could'nt be the case and because some of my
apps need to handle up to hundered of connections peer second in "write
to PostgreSQL db mode", i could'nt use MC/Rev server-sided stuffs
anymore...
Here is a short resume (previously posted to the MC/Rev lists, in
different occasions) in about this CGI bug workaround proposal :
1.- The sockets listener/translator part, as "istream.php" in this
example :
> <?
>
> // if ($REQUEST_METHOD == POST) { // PHP < 4.10
> if (!empty($_POST)) { // PHP >= 4.10
>
> // $headers = $HTTP_POST_VARS; // PHP < 4.10
> $headers = $_POST; // PHP >= 4.10
>
> while (list($header, $value) = each($headers)) $exAE .=
> "$header=$value&";
> $exAE = urlencode($exAE);
> $activapp = substr($exAE,0,5);
>
> if ($activapp == "is001") {
>
> $connection = fsockopen("localhost", "9482", &$error_number,
> &$error_description, "30");
> if ($connection) {
> set_socket_blocking($connection, true);
> fputs($connection,"");
> fputs($connection,"$exAE"."\r\n");
> fpassthru($connection);
> }
> else {
> $connection = fsockopen("localhost", "948", &$error_number,
> &$error_description, "30");
> if ($connection) {
> set_socket_blocking($connection, true);
> fputs($connection,"");
> fputs($connection,"$exAE"."\r\n");
> fpassthru($connection);
> }
> else print("erreur numÈro $error_number
> ($error_description)<BR>\n");
> }
> }
> }
>
> else print("<HTML><HEAD><TITLE>ERROR: File Not Found</TITLE></HEAD>".
> "<BODY background='/gif/fond-maison.gif'><H1>File Not
> Found</H1><P>".
> "<H3>The file you have requested does not exist on this
> server.</H3>".
> "</BODY></HTML><BR>\n");
>
> ?>
2.- The MC/Rev stack's script server-sided app part :
> on newconnect s
> read from socket s for 1 line with message "serverread"
> end newconnect
>
> on serverread x,y
> global PostIn,Retour,Lepath,MajDesIndexs1
> put urldecode(y) into PostIn
> put cr into char (length(PostIn))-1 to (length(PostIn)) of PostIn
> put "" into Retour
> set itemdelimiter to "&"
> iStreamSwitch
> write Retour to socket x -- with message csocket
> close socket x # x = adresse IP & "|" & n° d'ordre du socket
> encapsulant l'échange de données
> repeat
> if the num of lines in (opensockets()) > 1
> then close socket line 2 of (opensockets())
> else exit repeat
> end repeat
> end serverread
>
> # on csocket x
> # close socket x # x = adresse IP & "|" & n° d'ordre du socket
> encapsulant l'échange de données
> # end csocket
>
> on socketTimout
> end socketTimout
>
> on PreOpenStack
> if the short name of this stack is not "istreamserver" then pass
> PreOpenStack
> global Lepath,dbUsers
> put " Please, wait. iStream app's server loading..." into fld
> "status"
> set twelveHourTime to false
> put "iStream app's server up since" && the short date & "," && the
> short time into binfo
-- snip --
> set the socketTimeoutInterval to "10"
> if the num of lines in the windows is "1"
> then
> accept connections on port "948" with message "newconnect"
> put "with proxying keycode 3 actived..." into thep
> else
> accept connections on port "9482" with message "newconnect"
> put "with proxying keycode 7 actived..." into thep
> end if
-- snip --
> if the result is not "" then
> open file Lepath & "iStreamLog.txt" for append -- 2
> write cr & the result & cr to file Lepath & "iStreamLog.txt"
> put " " & the result after Pboot
> else
> open file Lepath & "iStreamLog.txt" for append
> write return & return & "Open sockets :" && the opensockets &
> return & return to file Lepath & "iStreamLog.txt"
> end if
> close file "iStreamLog.txt"
> replace " eof" with "" in Pboot
> if the num of words in Pboot > 0 or DbUsers is ""
> then put "iStream app's server not up :" & Pboot into fld "status"
> else put binfo && thep into fld "status"
> put the files into ERBList
> filter ERBList with "*ERB.rev"
> repeat for each line l in ERBList
> if l is not in the stacksinuse then start using l
> end repeat
> insert the script of stack the short name of this stack into back
> end PreOpenStack
3.- Your web or Rev's app "POST" request...
Hope this can help,
Kind Regards, Pierre
Le 8 avr. 04, à 07:13, Sannyasin Sivakatirswami a écrit :
> I changed the title of this thread to more accurately reflect the real
> issue as we think we have discovered it to be:
>
> Failure of the Rev CGI Engine to properly accept data from a POST by a
> Rev Client except with very
> specific syntax....
>
> This is probably something we need to carve into stone to save future
> CGI users a lot of grief.
> And, its quite bizarre and unintuitive. And we still don't know the
> deep causes behind it, only a solution.
>
> further examination of the apache error logs showed
>
> 192.168.1.246 - sevak [07/Apr/2004:17:53:08 -1000] "POST
> /cgi-bin/uploadGitaTranscript.cgi HTTP/1.1" 200 -
>
> There should be a length value at the end of that string e.g. 200 -
> 1158
>
> The absence of the length value indicates that, apparently, apache is
> not getting any POST data at all into stdIn from the Rev client app...
> but our sys admin here was thinking that his Apache and OSX server
> were solid and he ask me to try to post the same url encoded string
> from the terminal using
>
> curl -k
> "http://user:pass@our.intranet.web.site.org/cgi-bin/niftyRev.cgi" -d
> "long urlEncoded string of name=value pairs here"
>
> Ok, so in this case the POST is coming from the terminal --> LAN -->
> OSX server --> apache --> Revolution darwin engine:
>
> it worked every time!
>
> so, this would at first have indicated there was a problem with the
> Revolution client app sending the POST string. But the same client app
> works fine, posting to the same Rev cgi on a solaris machine where the
> engine is an old Metacard engine
>
> Then we dug out an old note from Scott Raney on this subject of POST
> failures where there was a lot of data being sent... but we had been
> experiencing this even on very small data uploads... and the syntax
> in the CGI he sent for the fix reads:
>
> put empty into tIncomingData
> repeat until length(tIncomingData) >= $CONTENT_LENGTH
> read from stdin until empty
> put it after tIncomingData
> end repeat
>
> Jai Ganesha! That works... so, it is some very esoteric, obscure issue
> about the way POST is being generated by the Rev client that requires
> this specific mode of reading stnIn in order to work, even though if
> you just used
>
> read from stnin untiil empty
> put it into tIncomingData
>
> it *will* work just fine if the POST string is sent by some other
> remote agent besides a remote Revolution app...
>
> but the following does *not* work.
>
>> repeat while length(tIncomingData) <= $CONTENT_LENGTH
>> read from stdin for $CONTENT_LENGTH
>> put it after tIncomingData
>> end repeat
>
> Don't ask "why" ;-)
>
> Jeanne... can you add this to the documentation for CGI side
> scripting? And, if Scott Raney is still in the loop we would love his
> input on this.
>
> Sannyasin Sivakatirswami
> Himalayan Academy Publications
> at Kauai's Hindu Monastery
> katir at hindu.org
>
> www.HimalayanAcademy.com,
> www.HinduismToday.com
> www.Gurudeva.org
> www.Hindu.org
>
> On Apr 7, 2004, at 12:37 PM, Sannyasin Sivakatirswami wrote:
>
>> Good intuitive sleuthing, David!
>>
>> OK I added the expected variable to tResponse being sent back and
>> that's very helpful.. what is happening, in either mode of reading
>> stndIn... is that the variable that should contain the incoming data
>> is completely empty! but, *occasionally* it does work.
>>
>> Bottom line now seems to be: POST is failing completely on most
>> occasions, either in the client side Rev App which is posting an
>> empty string... or on the Darwin side server CGI interpreter which is
>> failing to read anything at all from stnIn from the client side Rev
>> App. a) it NEVER works if I use the "repeat while len() form b) it
>> works But it does work *sometimes* Arghh! ;-)
>>
>> Whether I do either of these:
>>
>> repeat while length(tIncomingData) >= $CONTENT_LENGTH
>> read from stdin for $CONTENT_LENGTH
>> put it after tIncomingData
>> end repeat
>>
>> put tIncomingData into tCheckRawData
>> put urlDecode (tIncomingData) into tDataIn
>>
>> --[or:]
>>
>> --read from stdin until empty
>> --put it into tCheckRawData
>> -- put urlDecode (it) into tDataIn
>>
>> I get the same result
>>
>> can't open file
>>
>> /remote-team/ ## this shows that both the tUser and tFilename are
>> empty
>>
>>
>> I also added a data check to put the entire string of urlEncoded data
>> into a variable to test at the end and return in the response like
>> this:
>>
>> put the Result & cr & cr & tFilePath & cr & cr & tCheckRawData & cr &
>> cr into tResponse
>>
>> If I use the "repeat while len..." etc style, tCheckRawData contains
>> the literal variable "tIncomindData" which I assume means that
>> tIncomindData is, as a container, empty.
>>
>> if I use the simple 'read from stnin until empty; put it into
>> tCheckRawdata"
>>
>> the response line that should contains that data is completely empty.
>>
>> Conclusion at this point:
>>
>> POST is failing completely on most occasions, either in the client
>> side Rev App which is posting an empty string... or on the Darwin
>> side server CGI interpreter which is failing to read anything at all
>> from stnIn from the client side Rev App. Arghh! ;-)
>>
>> Ok, what now? I'll move this whole scenario over to our virtual host
>> site at mahiai.aloha.net where we have an old version of MC running
>> on a solaris... then if it works there we kind of have it isolated to
>> the current versions of Rev... OR OSX server version of Apache or
>> Darwin?
>>
>> Any more suggestions? I REALLY need this to work as our server admin
>> has turned off FTP and now this is the only way I can get data in
>> from the outside.
>>
>>
>>
>>
>> On Apr 6, 2004, at 9:23 PM, Dave Cragg wrote:
>>
>>> At 6:49 pm -1000 6/4/04, Sannyasin Sivakatirswami wrote:
>>>
>>>> Simply stated: the CGI accepts incoming data from a POST and writes
>>>> it to a file sometimes and sometimes it says it can't open the file.
>>>> No pattern...
>>>>
>>>> CONTEXT:
>>>
>>> <snip>
>>>>
>>>> the CGI (truncated a bit for email purposes:) goes like this:
>>>>
>>>> on startup
>>>>
>>>> if $REQUEST_METHOD is "POST" then
>>>> read from stdin until empty
>>>> put urlDecode (it) into tDataIn
>>>> split tDataIn by "&" and "="
>>>> put tDataIn["_remotestaff"] into tUser
>>>> put tDataIn["_project"] into tProject
>>>> put tDataIn["_transcript"] into tTranscript
>>>> put tDataIn["_fileName"] into tfileName
>>>> put url "file:transcriptionTeam.txt" into tAuthenticate
>>>> if (tUser is among the lines of tAuthenticate) then
>>>> # set up a file path to the incoming transcription
>>>> # it will just be a small xml file
>>>> switch tProject
>>>> case "taka"
>>>> put "/taka/New-Not Yet Posted/" into tLocalDestination
>>>> break
>>>> case "gita"
>>>> put "/gita/new_incoming/" into tLocalDestination
>>>> break
>>>> end switch
>>>> put ("../remote-team/" & tUser & tLocalDestination & tFileName)
>>>> into tFilePath
>>>>
>>>> # next: open, write data and close the file
>>>> ## !! but here is the problem:
>>>> ## this attempt to write a file fails intermittently...
>>>> # sometimes apache writes the file
>>>> # other times it returns "can't open file" to the result
>>>>
>>>> put tTranscript into url ("file:" & tFilePath) ## fails
>>>> intermittently
>>>>
>>>> put the Result & cr & cr into tResponse # sometimes empty some times
>>>> "can't open file"
>>>>
>>>> ## send stuff back to the user to confirm, along with the result
>>>>
>>>> end start up
>>>>
>>>> What is happening is when then the user clicks the button in the
>>>> remote rev app, to trigger the upload to Kauai it may return result:
>>>> "can't open file" then he clicks again and this time gets no result
>>>> and the file is written.
>>>
>>> One thing you might want to check first is that the CGI is reading
>>> in all the data.
>>>
>>> repeat while length(tDatain) >= $CONTENT_LENGTH
>>> read from stdin for $CONTENT_LENGTH
>>> put it after tDatain
>>> end repeat
>>>
>>> I can't say for sure, but looking quickly at your code, it seems you
>>> might get the error you described if the tUser variable was in fact
>>> empty and the tAuthenticate variable contained an empty line. You
>>> could check this by returning tFilePath when you get an error to see
>>> if the file path is the one you expect.
>>>
>>> Cheers
>>> Dave
>>> _______________________________________________
>>> use-revolution mailing list
>>> use-revolution at lists.runrev.com
>>> http://lists.runrev.com/mailman/listinfo/use-revolution
>>>
>>
>> _______________________________________________
>> use-revolution mailing list
>> use-revolution at lists.runrev.com
>> http://lists.runrev.com/mailman/listinfo/use-revolution
>>
>
> _______________________________________________
> use-revolution mailing list
> use-revolution at lists.runrev.com
> http://lists.runrev.com/mailman/listinfo/use-revolution
>
>
--
Bien cordialement, Pierre Sahores
100, rue de Paris
F - 77140 Nemours
psahores (at) easynet.fr
GSM: +33 6 03 95 77 70
Pro: +33 1 41 60 52 68
Dom: +33 1 64 45 05 33
Fax: +33 1 64 45 05 33
Inspection académique de Seine-Saint-Denis
Applications et SGBD ACID SQL (WEB et PGI)
Penser et produire "delta de productivité"
More information about the use-livecode
mailing list