Revolution and Task (handler) Overlap - Questions

[Scott Raney]

On Sat, 9 Feb 2002 "Gary Dennis" <gary.dennis at mantissa.com> wrote:

> Thanks for the calculation (the reason for my original email). I didn't say
> that  the application would keep the data,only that it would receive and
> screen. As for the data spew, the 800 tasks might be intellectual property
> of  400 companies. It would be some task to contact all 400, tell them that
> I had written an application using a tool  that was incapable of managing
> the combined message flow and asking them to make changes in the messages
> their software produced to compensate for the less-than-stellar performance
> of my application.

Seems to me if they expect *you* to do the montoring, you not only
have the right to ask this, but the obligation.  Your systems sound
like they're all time-shared, making it unfair for one app to use more
than it's quota just because it was sloppily coded.  You oughta be
charging them per byte that you have to deal with.

> > My second thought: don't use a Windows operating system for something
> > that calls for a more robust solution.  You'll get much better
> > bandwidth through a Linux or BSD box (or any other UNIX box, for that
> > matter) in this type of application.
> 
> Recall that the development  tool is Revolution.  Develop anywhere, run
> anywhere; right? Thirty one years in software development tells me that you
> develop on the least robust target platform, tune for the worst,  and hope
> that the customer can  and will make the switch when the volume dictates.
> If some of your following answers are true for all platforms, what real
> difference would Linux make?

Probably not much, maybe 30-50% increase in bandwidth.  But if you
really want to take your own advice, you should develop on MacOS 8/9,
which has the worst networking bandwidth ;-)

But you're right that a primary benefit of RR/MC is that you can
easily switch to another platform if it turns out the one you're
developing on isn't up to the demands you're placing on it.

> My goal is to make a decision on a cross-platform development tool for my
> company not  become an OS propeller head. I just watched 2 guys with 8 years
> combined Unix/Linux experience mud  wrestle with a portable, SUSE Linux, and
> a  $65 PCMCIA card for two days before getting it to connect to the LAN.
> Life is too short, then you die.

Very true: obscure hardware is definitely not Linux's strength.  But
this is not the typical server-environment problem, security,
reliability, and performance are (and all three of which are where
UNIX is IMHO stronger than any Windows OS).

> > > At this point in the development, all the test case product actually did
> was
> > > a.) get the message buffer, b.) break the buffer down into lines (crlf
> > > delimited message lines, the lf corresponds to the Rev RETURN, the cr
> part
> > > of the message is discarded) and c.) place the  message lines AFTER a
> > > scrollable field defined on a card.  After a  line limit is reached (set
> at
> > > 3000 lines), the handler would delete the FIRST line in the field.
> >
> > Another suggestion: if you're really dealing with a lot of rapidly
> > changing data, don't even put all of it in fields.  Just put the part
> > you need displayed in there and keep the rest in a variable.  Use a
> > scrollbar object to allow the user to select what part is shown.
> 
> What am I missing?  The field I'm using is scrollable.  The idea was to give
> the user a view anywhere inside the "retained" number of lines. Is there
> some rev object that will give me live scrolling of variable content?

No, you'd need to script that.  Whether you need to do this
optimization depends entirely on the amount of stuff you're putting
into the field and how often you're changing it.

> > > Question1.  Is there a mechanism in Rev that causes all messages to be
> > > stacked and executed in sequence as the targer  handler exit is taken
> for
> > > the previous message?  I had assumed that this is the case.  Is there an
> > > exception to this?  Can you take a message off the message stack for a
> > > handler and requeue it?
> >
> > Yes, no, and no.
> 
> So once you send a message, that message is handled next regardless of other
> messages that may be received?

If you use plain "send", it's processed immediately.  If you use "send
.. in", it'll be put at the *end* of the queue, so all other events
with the same time will get processed first.  Messages that are
generated from sockets have an effective delay of 0, but still are
processed in the order they're received.

> > This just seems like extra work to me.  If the CPU has time to do it
> > later (in addition to the extra overhead), why wouldn't it have even
> > more time just to handle the data as it comes in?  The only possible
> > disadvantage I can see would be that you might get several "messages"
> > in one read, but it seems to me this is likely to happen in any case
> > so you've got to handle it anyway.  Normal TCP buffering should take
> > care of intermittent (lasting a second or two) overloads.  If you
> > really do need to do some "batch" processing on the data that takes a
> > lot of CPU time, you should start thinking about offloading it to a
> > different machine or at least to a different, lower priority, process
> > if you must use only a single machine.
> >
> 
> It wouldn't have more time later. This would be  especially  true if the
> read is a BLOCKING read.  If it is tell me now and I  will give our copy of
> REV  to charity ( though that wouldn't be very charitable). 99 % of the time
> we get more than one message.  One operator command issued on such a
> mainframe could generate 16 to 32K of message output.  The advantage of the
> send (if and only if  the read is not a blocking read) could be that the
> read wait time could be used by handler that was the target of the send. I'm
> trying to figure out if  REV is a popgun UI tool that I'm trying to use for
> Rhino.  Synchronous thinking in what has to be an async environment won't
> really buy me anything.

If you use "read ..  with message", it'll be async, and the message
will be sent when the read completes.

> > > Question 2.  Does this approach give you any true overlap in the
> > > receive-process cycle?  It appeared to do so.  My assumption is that the
> > > engine is multithreaded in all cases with calls back to the engine
> providing
> > > the opportunity for handler overlap.
> >
> > Sockets are no longer threaded as of engine version 2.4.1
> > (subprocesses on Win32 are the only remaining threaded code in the
> > engine, and that's only because the Win32 API is missing some key
> > functionality in this area).  We ripped all the threading stuff out
> > because it was slow, significantly increased memory requirements, and
> > was just less reliable in general.  The apparent overlap is just
> > because you're moving stuff from the front of the message queue to the
> > end.  But this operation isn't free, and just makes processing take
> > longer overall.  The more things you queue up with "send .. in", the
> > worse the problem gets (keep in mind that it's got to sort through
> > that queue each time through the main event loop).
> 
> No longer threaded for Win32 or for all target platforms? Slower for Win32
> or all platforms?

All platforms, all platforms.

> Let me get this straight. You simply push the message to the end of the
> queue so that...
> 
> 
> read from socket x ... messagereceived
> on messagereceived s,data
>      send " messageprocessor  data"

I don't see the point of using "send" here.  Why not just process the
data in that handler, or call another function in the same script to
do it?

>      read from socket s .... messagereceived
> end messagereceiver
> 
> causes the sent message  put to be pushed to the back of the queue until the
> read is satisfied?

No, the message is sent immediately.

>  Is the read truly a blocking read?   If so, is it that
> way for all platforms?

If you're using "with message", it's async, but that's completely
different from the "send" in your example.

> Slow is relative, significantly increased memory would be expected, and just
> less reliable would usually relate to the quality of the code.

Maybe, but it's *always* the case that for a given amount of
development and QA time, a multithreaded app will be less reliable
than a non-threaded app.  This is one of the dirty little secrets of
buzzword-compliant software development.

>  Please tell
> me that this was not done for all target platforms.  We had planned to
> recommend deployment under OSX.  If this was done for all platforms, it
> strikes me that you had some stellar goals that you gave up on.

We just expected it to work with decent performance and reliability.
To be honest, 90% of the problems we had with threading were on the
Mac, where threading is "cooperative" (aka "a joke").  But we did have
a few really nasty race-condition type problems on Win32 which are
just standard behavior for multithreaded development which is why we
dropped threading there too.  And as I said before, performance
increased and memory usage decreased as a bonus.

> > > Question 3.  Is there some way to cause Rev to execute one handler at a
> > > higher priority than other handlers?
> >
> > No.
> >
> Seriously?  Not even the ability to specify that execution  is ASYNC as it
> relates to other handlers?

The only distinction is between "send .. in" and "read .. with
message" vs. those same commands without the additional parameter
specifying async operation.

> > PS: I'd also recommend sending your WWW site designers out for
> > "reeducation": I find it personally offensive that you can't even view
> > www.mantissa.com except on a Windows system (it won't even open with
> > browsers other than IE, and is all garbaged up on Macs even with IE).
> > You'll not likely be getting any business from us, or any other users
> > of UNIX or Mac systems, if you stick with this policy.
> >
> 
> We design and write software, not web sites.

I guess you should probably just shut down the site entirely, seeing
as how it isn't one of your "core competencies".  As a bonus, you'd
save some money too.

> Besides, if we had everything
> (including web designers), where would we put it (or them)? If we did sell
> our products  to UNIX, Linux,  or non-i.e. customers we would owe them a
> refund since we have no such products. However, if  those platforms are ever
> supported, we will change the site. Until then, whenever you want to be
> personally offended please visit our site.

Hey, you never know where your customers are going to be coming from.
We may not be needing whatever service you're offering, but then when
one of our customers asks for a recommendation for a particular kind
of service (which happens about every day), I won't be recommending
yours because despite trying to find out, I still don't know what it
is.  Worse, if your company's name happens to come up, my first
response is going to be how poor its understanding of marketing
is and why that means it'd be a poor bet for long-term survival.
  Regards,
    Scott

********************************************************
Scott Raney  raney at metacard.com  http://www.metacard.com
MetaCard: You know, there's an easier way to do that...