suggestions for cross-version benchmarking test tool?
ambassador at fourthworld.com
Tue Oct 7 10:57:08 CDT 2008
Bernard Devlin wrote:
> Ok, so since Chipp thinks my benchmark is pretty crummy (which I'm happy to
> believe), can someone provide me with what they think would be a better way
> to benchmark the speed of the different engines? I'm reluctant to use
> Wilhelm's suggestion on Linux, since there is a known memory leak to do with
> the interaction of Rev and XWindows on Rev 2.9 and Rev 3.0 (by my
> observation Rev causes X to leak about 2mb every 3 seconds whilst the stack
> that demonstrates the leak runs).
I just searched the RQCC for "leak" and couldn't find that one. Of the
five results that came up in that search it seems only two have strong
evidence of being a leak, and both have workarounds provided to avoid
the seeming leak condition.
What is the stack doing while it's "leaking" "2mb every 3 seconds"?
That seems like a lot.
Seeing the scripts in question will be very helpful, esp. if submitted
in an RQCC report.
It wouldn't surprise me if a code base this complex has a leak or two,
but I've noticed that RunRev tends to give priority to leaks, perhaps
because of their insidious nature and that they're usually relatively
easy to fix.
> As Chipp suggested, I searched the archive for 'benchmarks gaskin' but
> didn't come up with anything that seemed like a test of the engine rather
> than a test of a specific feature e.g. regex, filter, offset. I had a look
> at Richard's revBench but that seems to be a tool designed to just compare
> two different scripts within the same version of Revolution.
True, much of my benchmarking is to find optimal ways to accomplish a
given task, independent of platform. There are so many ways to do
things in Rev, and I've found that sometimes what might seem the most
obvious may not offer the best performance.
One of the conveniences of RevBench (available in RevNet for those who
haven't seen it -- in Rev choose Development->Plugins->GoRevNet) is that
it can be saved so you can run the same tests across different platforms.
But it may be just as simple to just make a fresh stack that tests what
you need. What does the project you're working on do? Perhaps we might
be able to help make a good test for both benchmarking and honing in on
the possible leak.
> I'm looking for something to explain why there are noticeable flickers (for
> me) in various components of the Rev 3.0 IDE (elements within the menu bar
> and within property inspectors).
I hadn't noticed that until you brought it to my attention; seems I've
been running without icons on my Ubuntu system, so while it's still
present it isn't as noticeable. I turned on the icons, and yep, I can
see it now.
My hunch is that it's doing some redundant checking during boot, or
maybe building something dynamically behind the scenese which requires
changes to the editGroupControls. It seems limited to that one button,
and none of my own stuff seems affected, so it seems to be just a
scripting issue in the IDE, and a minor (cosmetic) one at that; the
functionality of that button seems unaffected.
Given what I can see of it, this flicker doesn't appear to be
engine-related and shouldn't affect any projects you make with Rev.
> I think that Rev 3.0 on Linux is just running slower than it is on Windows.
I've seen broad variance in performance across platforms. Last time I
did any testing of that sort I found that text parsing was much faster
on Windows than on Mac, even when the Windows installation was VirtualPC
running on a Motorola-powered Mac (!).
I asked Scott Raney about that at the time and he said that the biggest
difference he's seen for platform-independent tasks like text processing
(as opposed to system-specific tasks like object rendering) is with
compiler design: he said the MS compiler was just darn good at its job
relative to Code Warrior (which he was using at the time).
If there is a performance difference between Windows and Linux, I
suspect there might also be a similar difference between Windows and OS
X. I believe that XCode shares many elements in its compiler with GCC,
so it may be the case that benchmarking today would show similar
results, with Windows outperforming both OS X and Linux.
Some things may be optimizable, and I think Mark Waddingham would agree
that there's still much that can be done to refine the integration of
Rev's rendering engine with the Linux window manager interfaces.
But the differences in how the various OS families render windows and
controls are so vast that they don't make for good comparison of Rev
engine performance. For those task I would test things as Chipp
suggested, with the lockScreen true so we're focusing more on the
engine's internals and less on the host OS.
Of course, with several thousand tokens it wouldn't be practical to test
them all. What sorts of things are you working on? If start with what
you need to do at the moment we'll have not only an accomplishable
subset of testing tasks, but one which is especially relevant for your
Managing Editor, revJournal
Rev tips, tutorials and more: http://www.revJournal.com
More information about the use-livecode