Chrome, lchttpd and a possible Livecode socket bug?
ambassador at fourthworld.com
Wed Jul 15 17:21:43 CEST 2015
David Bovill wrote:
> @andre - dropbox is no longer public. I'd love to check out your old
Andre's been immersed in the Firefox project, and doesn't frequent this
list as he used to. Back when he did he wrote at least two lengthy
posts here about why he stopped development of his httpd stack, and why
he doesn't recommend using it for Web servers.
In short, serving browser clients is especially challenging in a
non-threaded and non-forking engine like LiveCode, because what we think
of as a single transaction is often a dozen or more - in addition to the
of images files, resulting in many requests for a single page.
Because the engine is single-threaded and non-forking, all requests are
effectively queued. Network I/O is somewhat non-blocking, but
everything else the engine does, such as file I/O, is blocking, so
handling things like Web pages with their multiple requests becomes more
onerous than just using any of the apps already doing a good job for
that, like Apache, NGineX, etc.
Using LC as a CGI under Apache is a reasonably good use case, as it
allows concurrency under Apache's forking. But replacing Apache with LC
is perhaps more trouble than it's worth, at least until we can do
That said, I do believe there's good value in exploring LiveCode daemons
for other uses beyond Web servers.
Application-specific servers, where transactions will be a single
request rather than a dozen or more as is common with Web pages, can be
very good fit for LiveCode. Even better when the client is also made
with LC, so we can take advantage of things like compressed encoded
arrays for much more efficient throughput than translating to/from JSON.
In that sense, whether we use port 80 with HTTP conventions or make a
protocol of our own over another port, we're making a simple socket
server, and in my tests LC does well with that sort of thing.
One of my favorite examples along these lines is EVE Online, since it's
one of the world's most popular MMOGs and is served by a single Python
In all fairness, though, since Python is like LiveCode in that it
doesn't support true concurrency, a special build of Python was needed,
CPython, which includes support for "green threads" that provide
concurrency but with less overhead than system-managed threads.
A similar enhancement could be made with LiveCode, and I hope it comes
about some day since it would allow us to do everything Python can do.
But in the here-and-now, few of us need to solve a C10k problem, and
with reasonable traffic a single LiveCode daemon can do well.
If traffic grows beyond what a single instance can handle, the
bottlenecks can be distributed among any number of other LC instances
managed by the daemon via local sockets, as we've discussed here before.
To explore the potential for building socket servers with LC, I've found
two projects especially helpful: mchttp and the chat example.
Both Andre and Todd started their httpd projects as forks of Scott
Raney's mchttpd. Dr. Raney's original stack no longer works because
most browsers today have stricter requirements for header info than his
script provided, but with his permission I added only Andre's header
fix to Raney's original script and the result is available under MIT
Andre's and Todd's enhancements are likely very useful; I'm providing
the link to Raney's fixed original only because if there seems to be an
issue in how the engine is handling sockets this simpler script may make
The chat client and server code in this lesson is about as simple as it
gets, making it a good starting point for expansion since it was part of
a client deliverable and is known to work well:
While the mchttp stack is a good example of simple transaction handling
in which a single client makes a request that is returned only to that
client, the chat example illustrates broadcasting in which any data
coming in from any client is broadcast to all of them.
Between these two models we have a nice foundation for a great many
possible types of interesting and useful client-server applications.
With the socket issue you've seen, it may be useful to apply the logging
you have to either mchttpd or the chat example and see what you find.
I've spent very little time with mchttpd, but I did some stress testing
on the chat server a while back and it held up admirably: I had three
separate client computers throwing as many requests as they could to the
daemon running on one of my VPSes for two minutes, monitoring progress
in htop as it ran. The daemon never used more than 25% of the CPU, and
when the barrage ended it reclaimed all memory and dropped back to zero CPU.
Hopefully you'll find a quick fix in one of the scripts, and if it does
indeed turn out to be an engine issue your bug report to the RQCC is
Fourth World Systems
Software Design and Development for the Desktop, Mobile, and the Web
Ambassador at FourthWorld.com http://www.FourthWorld.com
More information about the use-livecode