OT-ish: Dreamhost and LiveCode?

[Richard Gaskin]

Graham Samuel wrote:
 > Way back in May there was a discussion of hosting companies that were
 > reliable and LiveCode friendly. Dreamhost was picked out as a
 > good’un, but at the time there seemed to be some doubt about the
 > speed of LiveCode Server on the platform.

Most hosting companies support CGI, so just about any will do.  I like 
Dreamhost, but I also really like InterServer, and many here have also 
had a good time with JaguarPC, HostGator, and others.

Any host with multiple redundant high-speed connections, on-site 
generators, nightly backups, and other baseline core competencies will 
be a good choice, and these days there are few that don't have those basics.

The only thing that I've seen much variance in is that there are still a 
small number of hosts that don't support SSH.   This makes them unusable 
for any serious work beyond simple static pages, because even if you're 
not a big fan of Terminal sooner or later you'll need to do something on 
that machine that's either difficult or impossible to do with FTP alone. 
  With ssh you can log in and it's as good as sitting at the desk with 
the machine in front of you, but without it you're hosed.  So always 
insist on ssh.  It's a godsend, even if you only use it occasionally.


 > Can anyone say what the status is now? I’m looking for a US-based
 > LiveCode-friendly hosting organisation for a modest commercial web
 > presence (by ‘commercial’, I mean it will sell downloads).

That issue has since been clarified as not at all specific to Dreamhost. 
  To varying degrees it may be evident on any 64-bit system, which these 
days is pretty much all of them.

The symptom is that execution time of CGIs which had been measured in 
milliseconds using the v6.x engine were now taking as long as two or 
three seconds in some cases with the v7 engine.   This is much longer 
than anything attributable to Unicode or other areas where we've seen 
some performance degradation in v7, serious enough to merit specific 
attention since Web users have come to expect nearly instantaneous 
responses from modern server systems.

Phil Davis filed this bug report with LC, and initiated an investigation 
into the root cause with Dreamhost support:
<http://quality.runrev.com/show_bug.cgi?id=13983>'

Many kudos to Andrew and the three other Dreamhost engineers that lent a 
hand in pinning down the root cause, and to LiveCode's Mark Waddingham 
and Dr. Peter Brett for finding two solutions for it.

I'll outline the issue here, and briefly describe the current status of 
available solutions:


The issue isn't about 64-bitness per se, but that the 64-bit requirement 
means we can only use LC v7 or later, since v6.x and earlier are only 
available in 32-bit builds.

Among the many changes in v7 was improved support for Pango, the font 
subsystem used on Linux.  While these changes make for much better 
support for fonts on Linux, they've also added additional time to the 
boot sequence.   This additional time is small enough that it doesn't 
matter much in desktop apps, but CGI apps are by nature much more 
time-sensitive, since the engine is launched, runs, and dies with each 
request, so things that may not matter much on the desktop can matter a 
lot in a CGI environment.

Dreamhost and some other systems include a full set of fonts, and during 
startup LC obtains the location of the fonts folder and walks through 
each font file to initialize it for use.

This may seem unnecessary given that most CGIs only process text and 
never need to render fonts at all, but keep in mind that since v6.6 and 
later the server engine can now render graphics for some very 
interesting uses like customized postcards, diagrams, etc., so having 
access to fonts in that engine can potentially be every bit as useful as 
in a desktop app.

But the time required to init all those fonts adds up, so Waddingham and 
Brett came up with two solutions, a short-term workaround and some 
options for the long term:


Short-term workaround:

The trick is to tell LiveCode to use a different font folder than the 
one the system provides, ideally with only one simple font file in it.

Sounds easy enough in principle, but in practice it means creating a 
shell script to redefine the location of the font.conf file in the 
$FONT_CONFIG global, export that variable so child processes will use 
it, and then launch LiveCode as a child process.

With assistance from Waddingham and Brett I've been able to put this 
into place at livecodejournal.com, and the resulting performance is now 
on par with the earlier v6 setup I used to enjoy there.

I've been entrusted to document this workaround and provide the files 
needed to set it up, but client projects have been demanding so that 
article has had to sit on the back burner for a few weeks.  My 
apologies; I hope to have that completed soon.

If you find v7 CGI performance noticeably problematic in your 
applications, it would be helpful for me to gauge interest in this 
documentation effort, so please email me the URL to your affected web 
app at ambassador AT fourthworld.com .  If I get a lot of those requests 
I'll make an effort to prioritize that documentation accordingly, but it 
may also be that relatively few are affected, in which case maintaining 
a focus on keeping a roof over my head may not be a bad thing. :)  Happy 
to be of service if needed.


Long-term solution:

There are several options for modifying the engine's boot sequence to 
address this problem without requiring us to make cumbersome 
modifications to our site setup.

The simplest for us would be the hardest for the team: having the engine 
become smart enough in analyzing our scripts during boot so it can init 
only those things that are needed and bypass those that aren't.  So if 
you're using graphics output that will require fonts, it would do the 
font init, but if not it would skip it altogether.

However, that level of cleverness would take time to implement, so in my 
most recent comment at the bug report linked to above I proposed a 
middle path which would provide us with an easy setup while keeping the 
work of the engine team to a minimum:

Add an optional command line flag to turn off graphics initialization, 
including fonts.  I proposed using -g for this, similar to how we 
already have -ui for bypassing GUI initialization in the desktop engine.

For users this would mean that the subset of those who've noticed big 
slowdowns from font initialization could boost throughput by just adding 
that flag to their .htaccess directive.

For the team it means probably an hour or two's work to provide a 
solution that solve the problem well, while still leaving the door open 
for deeper changes they may consider later on.

I haven't yet heard back from the team on that proposal, but I hope 
they'll find it makes for a nice middle-path solution they can implement 
affordably and soon.

-- 
  Richard Gaskin
  LiveCode Community Manager
  richard at livecode.org