[revServer] process timeout issue

[Mark Talluto]

On Aug 4, 2010, at 7:53 AM, Richard Gaskin wrote:

> I've been experimenting with spidering, data mining, and analytics, and like any processor-intensive tasks it would never occur to me to put them on a shared host.
> 
> Like many hosts, the one I'm using offers dedicated servers for less than $70/mo, but being a cheapskate I've gone one step further during this experimental phase:  I bought a nettop off Ebay for just $150, set it up with Ubuntu and Rev, and that does all the heavy lifting 24/7, posting only the output from those process to my servers periodically as needed.
> 
> I never run into the CPU cycle limits most hosts have on their servers, and I don't even slow down my own web server from its tasks of serving pages to my visitors and handling their purchases.
> 
> When the workflow expands to required tighter integration between the processing and the output, I can move the system from my office to a dedicated server with multiple redundant fat-pipe connections for just a few bucks a month.
> 
> There are a million ways to create robust scalable infrastructures to handle any load.  Many are cheap and easy to do, and for most of those tasks you can do them all in one fun language.

We have been using this technique for years.  We even posted the application we use to do this task in RevNet.  I believe I need to update that file now that I think of it.  But in short, we use our ISP to gather orders.  Our client software sends a request for a key.  Our local computer in the lab just pings the directory on the ISP every 4 seconds and downloads all the orders in that given directory.  The heavy lifting and database work is done on a computer in the lab.  The key is then sent back up to the ISP where the client computer is checking in for the result of that work every 4 seconds.  The whole thing works out nicely and we keep our CPU usage low.


Best regards,

Mark Talluto
http://www.canelasoftware.com