"fork" command?

[Richard Gaskin]

Monte Goulding wrote:

 >> On 8 Jan 2016, at 6:03 am, Richard Gaskin wrote:
 >>
 >> I think we need a new command that launches a specified process
 >> but in a way that uses a call to "fork" to pass file descriptors
 >> (which include sockets and other I/O info) to the child process.
 >>
 >> In many ways it would work very similarly to the existing "open
 >> process", but allow params to give the child process access to
 >> things like socket connections, pipes, files, etc. the parent
 >> process has access to at the time the child process is launched.
 >>
 >> It would seem least intrusive on the code base to implement it as
 >> a new command, perhaps called "fork".
 >
 >
 > Hmm… I’m not convinced we need this for FastCGI. Apache mod_fcgid
 > will start up processes for you and for Nginx and other servers
 > that don’t do that you could use spawn-fcgi.

My understanding is that spawn-fcgi uses fork, no?


 > We mainly need two things for FastCGI:
 >  - an engine with the FastCGI accept loop as the main loop (LC Server
 > just starts up and quits at the end of the code and standalones just
 > keep looping until you quit).

I believe that has more to do with the nature of CGI than with LC per 
se.  That is, as a CGI any engine (Perl, Python, Ruby, LiveCode) will be 
born, live, and die during the request.

But with FastCGI the engine is only loaded once, and instances forked 
with requests as needed, and using fork they get the socket and other 
data needed for the child process to handle the task.

Some engines may use multithreading rather than multiprocessing, but the 
difference is less of a concern on Linux than on Windows since Linux 
spawns processes much more efficiently.

If multithreading were pursued as an alternative to multiprocessing via 
fork, I fear a threading subsystem would be much more work to implement, no?


 >  - to decide on how to handle things like global variable scope etc
 > because you’re going to end up with multiple requests to the same
 > environment.

How is that handled in the FastCGI version of PHP?  I would imagine it 
would be no more onerous than with threading, arguably simpler since so 
much of the action takes places in a separate process.

I wouldn't expect to be able to use FastCGI without modifying some of my 
scripting habits; as with any new feature, just a few new things to 
learn and keep track of.  Indeed, I would welcome the opportunity for it 
to become possible to learn those things.


 > After some thought and considering the new script only stack format
 > I came to the conclusion that it would be better not to do the php
 > style <?lc tags and just use a script only stack with messages:
 >  - startup - the process is started do anything you can do once for
 >    all requests
 >  - acceptRequest - a new request needs to be handled and the global
 >    environment variables $_SERVER etc have been updated where
 >    necessary (or these could be parameters I guess)
 >  - shutdown - something is killing the process so do anything you
 >    need to do to cleanup
 >
 > The basic idea is you can do something like this:
 >
 > local sCounter = 0
 > local sProcessStarted
 >
 > on startup
 >   put the seconds into sProcessStarted
 > end startup
 >
 > on acceptRequest
 >   add 1 to sCounter
 >   put “Started:” && sProcessStarted & return & “Counter:” && sCounter
 > end acceptRequest
 >
 > If you only spawn one process then each time you hit the server the
 > counter would increment.

In that outline would "acceptRequest" be a request from Apache, or are 
you proposing a system that replaces Apache to accept requests directly 
from the client?

Once we have forking we could completely replace Apache (or NGineX or 
Node.js) with a fully-functioning server for specific applications where 
the efficiencies of a purpose-built system would be helpful.

But even when running under Apache with FastCGI, fork would seem a very 
useful thing.  It's how PHP and other engines are able to scale, and 
indeed not having it prevents LC from being used in traffic-heavy scenarios.


 > Of course you could have a FastCGI engine that cleared all the
 > globals and stacks from memory between requests and loaded any
 > script only stack file but it’s not quite as much fun, you lose
 > the advantage of keeping resources in memory and as far as I can
 > tell it’s a bit more work to do that ;-)

As with other persistent systems like LC on the desktop, we should 
maintain control over which data is purged and which data is shared.  We 
have globals and script-locals, depending on the context we need them, 
and in a multiprocessing environment we should have the same flexibility.

For example, one of the strong advantages of FastCGI or other persistent 
implementation is that we don't have to create and destroy database 
connections with every request.  That sort of information (along with 
config data and other such things) we'd want to remain globally 
available to child processes.  Request-specific data could be handled in 
script-locals, where they can be managed and cleared as needed within 
the worker process itself, without affecting truly global data managed 
by the parent.

-- 
  Richard Gaskin
  Fourth World Systems
  Software Design and Development for the Desktop, Mobile, and the Web
  ____________________________________________________________________
  Ambassador at FourthWorld.com                http://www.FourthWorld.com