suggestions for cross-version benchmarking test tool?

[Richard Gaskin]

Bernard Devlin wrote:

> Ok, so since Chipp thinks my benchmark is pretty crummy (which I'm happy to
> believe), can someone provide me with what they think would be a better way
> to benchmark the speed of the different engines?  I'm reluctant to use
> Wilhelm's suggestion on Linux, since there is a known memory leak to do with
> the interaction of Rev and XWindows on Rev 2.9 and Rev 3.0 (by my
> observation Rev causes X to leak about 2mb every 3 seconds whilst the stack
> that demonstrates the leak runs).

I just searched the RQCC for "leak" and couldn't find that one.  Of the 
five results that came up in that search it seems only two have strong 
evidence of being a leak, and both have workarounds provided to avoid 
the seeming leak condition.

What is the stack doing while it's "leaking" "2mb every 3 seconds"? 
That seems like a lot.

Seeing the scripts in question will be very helpful, esp. if submitted 
in an RQCC report.

It wouldn't surprise me if a code base this complex has a leak or two, 
but I've noticed that RunRev tends to give priority to leaks, perhaps 
because of their insidious nature and that they're usually relatively 
easy to fix.


> As Chipp suggested, I searched the archive for 'benchmarks gaskin' but
> didn't come up with anything that seemed like a test of the engine rather
> than a test of a specific feature e.g. regex, filter, offset.  I had a look
> at Richard's revBench but that seems to be a tool designed to just compare
> two different scripts within the same version of Revolution.

True, much of my benchmarking is to find optimal ways to accomplish a 
given task, independent of platform.  There are so many ways to do 
things in Rev, and I've found that sometimes what might seem the most 
obvious may not offer the best performance.

One of the conveniences of RevBench (available in RevNet for those who 
haven't seen it -- in Rev choose Development->Plugins->GoRevNet) is that 
it can be saved so you can run the same tests across different platforms.

But it may be just as simple to just make a fresh stack that tests what 
you need.  What does the project you're working on do?  Perhaps we might 
be able to help make a good test for both benchmarking and honing in on 
the possible leak.


> I'm looking for something to explain why there are noticeable flickers (for
> me) in various components of the Rev 3.0 IDE (elements within the menu bar
> and within property inspectors).

I hadn't noticed that until you brought it to my attention; seems I've 
been running without icons on my Ubuntu system, so while it's still 
present it isn't as noticeable.  I turned on the icons, and yep, I can 
see it now.

My hunch is that it's doing some redundant checking during boot, or 
maybe building something dynamically behind the scenese which requires 
changes to the editGroupControls.  It seems limited to that one button, 
and none of my own stuff seems affected, so it seems to be just a 
scripting issue in the IDE, and a minor (cosmetic) one at that; the 
functionality of that button seems unaffected.

Given what I can see of it, this flicker doesn't appear to be 
engine-related and shouldn't affect any projects you make with Rev.


> I think that Rev 3.0 on Linux is just running slower than it is on Windows.

I've seen broad variance in performance across platforms.  Last time I 
did any testing of that sort I found that text parsing was much faster 
on Windows than on Mac, even when the Windows installation was VirtualPC 
running on a Motorola-powered Mac (!).

I asked Scott Raney about that at the time and he said that the biggest 
difference he's seen for platform-independent tasks like text processing 
(as opposed to system-specific tasks like object rendering) is with 
compiler design:  he said the MS compiler was just darn good at its job 
relative to Code Warrior (which he was using at the time).

If there is a performance difference between Windows and Linux, I 
suspect there might also be a similar difference between Windows and OS 
X.  I believe that XCode shares many elements in its compiler with GCC, 
so it may be the case that benchmarking today would show similar 
results, with Windows outperforming both OS X and Linux.

Some things may be optimizable, and I think Mark Waddingham would agree 
that there's still much that can be done to refine the integration of 
Rev's rendering engine with the Linux window manager interfaces.

But the differences in how the various OS families render windows and 
controls are so vast that they don't make for good comparison of Rev 
engine performance.  For those task I would test things as Chipp 
suggested, with the lockScreen true so we're focusing more on the 
engine's internals and less on the host OS.

Of course, with several thousand tokens it wouldn't be practical to test 
them all.  What sorts of things are you working on?  If start with what 
you need to do at the moment we'll have not only an accomplishable 
subset of testing tasks, but one which is especially relevant for your 
current work.

-- 
  Richard Gaskin
  Managing Editor, revJournal
  _______________________________________________________
  Rev tips, tutorials and more: http://www.revJournal.com