read from socket question

kee nethery kee at kagi.com
Tue Oct 10 13:27:21 EDT 2006


I am using Andre Garzia's wonderful RevHTTPd code to create a small  
slave server to provide a service for other master servers. In  
general it works well except, one of the master servers does  
something funny but totally legit when it sends a POST to the slave  
RevHTTPd server and most times the RevHTTPd server deals with it, and  
sometimes not. This is a read from socket question and I could use help.

Short problem summary:

Master server sends the HTTP POST in two packets, one with the POST  
line and then the rest of the data in the second packet. Most of the  
time Revolution combines the two packets together and then asks my  
code to process it. Sometimes it just takes the POST packet and  
thinks that is all it is going to receive and asks my code to parse  
that. The problem is that the second packet of data is ACKed by  
Revolution but that second packet of data is not provided to me so  
all that data is lost. I need read from socket to give me that second  
packet of data 100% of the time.

Long problem summary:

The slave server code is very simple:

on httpStart
   accept connections on port 8080 with message "newConnection"
end httpStart

on newConnection pSocketID
   read from socket pSocketID with message "processRequest"
   -- message is actually doing a "processRequest pSocketID,it" where  
it contains all the data gathered by the socket.
end newConnection

on processRequest pSocketID, pSocketData
   global theSocketData
   put pSocketData into theSocketData
   -- then from here on down it parses theSocketData and does a reply  
to the master server


Here's the problem:

The master server (192.100.200.10:4698) talks to a RevHTTPd server  
(192.100.200.20:8080).

When it talks to it, it sends one line of POST information, waits for  
an ACK, and then sends the data. What happens is that most times the  
RevHTTPd server gathers the POST packet and the data packet and  
combines them together into the socketdata that gets processed. But  
sometimes it thinks the POST packet is all the data there is and it  
sends just that into socketData to get processed.

Needless to say, I need both packets of data to process the request.

I have a System 9 AppleScript that speaks to the same RevHTTPd server  
and it sends one packet that combines the POST data and the Data data  
so that they are always 1 packet and always included in the  
socketdata. That connection works 100% of the time because Revolution  
gets everything in one packet.

I've tried a whole bunch of things to get that second packet of data  
and so far nothing has worked. I tried seeing if a second message got  
passed with the Data portion after the POST portion came through but  
amazingly enough, that second packet is completely ignored when it  
thinks the first POST packet is all that there is.

Any ideas what I can do to have RevHTTPd not hang give up so quickly?  
Any idea why it happens sometimes and not others?

So far I can not see any difference between the ones that succeed and  
the ones that fail.

The timing between the POST and DATA packets are in the milliseconds  
and for POSTs that succeed and POSTs that fail, there is no pattern.  
Both can have short or long times between the packets.

I've tried a bunch of variations with reading for a bunch of  
characters, reading until a string of characters that is never going  
to be sent, reading to capture the second packet separately, so far  
none of these attempts have worked. I've grabbed the packet data from  
a failed session and shown it below.

Below is a set of data showing a failed communication.

-- master server contacts RevHTTPd and passes it the POST data

Packet received at 1160437657.030018
TCP packet from 192.100.200.10:4698 to 192.100.200.20:8080 (32 bytes)
IP Header:
  Version: 4; Header Length: 20; TOS: 0; Packet Length: 72
  Identifier: 46318; Do Not Fragment; Fragment Offset: 0
  Time To Live: 128; Protocol: 6; Header Checksum: 49569
TCP Header:
  Sequence Number: 2668230889; Acknowledgment Number: 4005153850
  Header Length: 20; Code Bits: ACK PUSH; Window: 8760
  Checksum: 35102; Urgent Pointer: 0
Data:
POST«spc»/intergraphicsD2«spc»HTTP/1.0«cr»«lf»

-- RevHTTPd ACKs (this happens whether it fails or succeeds, the ACK  
is the same)

Packet received at 1160437657.030069
TCP packet from 192.100.200.20:8080 to 192.100.200.10:4698 (0 bytes)
IP Header:
  Version: 4; Header Length: 20; TOS: 0; Packet Length: 40
  Identifier: 35601; Do Not Fragment; Fragment Offset: 0
  Time To Live: 64; Protocol: 6; Header Checksum: 11167
TCP Header:
  Sequence Number: 4005153850; Acknowledgment Number: 2668230921
  Header Length: 20; Code Bits: ACK; Window: 65535
  Checksum: 47118; Urgent Pointer: 0

-- Right here is where RevHTTPd either gives up and says it has  
gotten all the socketData there is to get, 
-- OR it adds the next packet to the socketData stream and then it  
sends that combined socket data to my code.

-- master server sends the rest of the data (again the same whether  
it succeeds or fails)

Packet received at 1160437657.031270
TCP packet from 192.100.200.10:4698 to 192.100.200.20:8080 (796 bytes)
IP Header:
  Version: 4; Header Length: 20; TOS: 0; Packet Length: 836
  Identifier: 46574; Do Not Fragment; Fragment Offset: 0
  Time To Live: 128; Protocol: 6; Header Checksum: 48549
TCP Header:
  Sequence Number: 2668230921; Acknowledgment Number: 4005153850
  Header Length: 20; Code Bits: ACK PUSH; Window: 8760
  Checksum: 9701; Urgent Pointer: 0
Data:
Content-type:«spc»application/x-www-form-urlencoded«cr»«lf»
Content-length:«spc»724«cr»«lf»
«cr»«lf»
ACG:ProductName=CSLDisco2&ACG:PurchaserName=Andreandre 
+Soapdog&ACG:SQNM=1352&A
CG:Request=Generate&ACG:UserPurchaseDate=2006-10-09&ACG:TransactionID=CH 
1234PY
5678&ACG:LanguageAndCharSet=English% 
2FASCII&ACG:LicenseType=single&ACG:DebugFl
ag=0&ACG:UnitPayment=25.000&ACG:PurchaserName-8bit=Andreandre 
+Soapdog&ACG:Inpu
tVersion=0200&ACG:CardName=Andreandre+Soapdog&ACG:Postal-8bit=%231 
+Appple+Way+
%23759-8%2C+South+San+Francisco+CA+94070%2C+United 
+States&ACG:CardName-8bit=Vi
ctoriano 
+Soapdog&ACG:TimeStamp=I531422589&ACG:ProductName-8bit=CSLDisco2&ACG:P
urchaserEmail=sdproductions%40comcast.net&ACG:Postal=%231+Appple+Way+% 
23759-8%
2C+South+San+Francisco+CA+94070%2C+United 
+States&ACG:DateProcessed=2006-10-09&
ACG:QuantityOrdered=1&

-- Again, the RevHTTPd ACKs (same either way)

Packet received at 1160437657.031347
TCP packet from 192.100.200.20:8080 to 192.100.200.10:4698 (0 bytes)
IP Header:
  Version: 4; Header Length: 20; TOS: 0; Packet Length: 40
  Identifier: 35602; Do Not Fragment; Fragment Offset: 0
  Time To Live: 64; Protocol: 6; Header Checksum: 11166
TCP Header:
  Sequence Number: 4005153850; Acknowledgment Number: 2668231717
  Header Length: 20; Code Bits: ACK; Window: 64904
  Checksum: 46953; Urgent Pointer: 0

-- But now RevHTTPd replies back and in this example, the failure, it  
only passed the first packet to
-- my code, the second packet with the Data in it was ignored.

Packet received at 1160437658.487054
TCP packet from 192.100.200.20:8080 to 192.100.200.10:4698 (285 bytes)
IP Header:
  Version: 4; Header Length: 20; TOS: 0; Packet Length: 325
  Identifier: 35625; Do Not Fragment; Fragment Offset: 0
  Time To Live: 64; Protocol: 6; Header Checksum: 10858
TCP Header:
  Sequence Number: 4005153850; Acknowledgment Number: 2668231717
  Header Length: 20; Code Bits: ACK PUSH; Window: 65535
  Checksum: 34135; Urgent Pointer: 0
Data:
HTTP/1.0«spc»503«spc»Service«spc»Unavailable«cr»«lf»
Content-Type:«spc»text/html«cr»«lf»
Content-Length:«spc»201«cr»«lf»
«cr»«lf»
«cr»«lf»
<HEAD><TITLE>intergraphicsD2«spc»Error</TITLE></ 
HEAD><HTML><BODY>There«spc»
is«spc»a«spc»data«spc»file«spc»named«spc»'intergraphicsD2.list'«spc»
but«spc»the«spc»quantity«spc»detected«spc»was«spc»not«spc»
quite«spc»right.«spc»So«spc»I«spc»am«spc»telling«spc»you«spc»
to«spc»retry.</BODY></HTML>


Any ideas?

I tried to find something that defines the criteria that read from  
socket uses to determine when a socket has gotten all there is to get  
and that is not defined anywhere as far as I can tell.

Kee




More information about the Use-livecode mailing list