Making Revolution faster with really big arrays

[Mark Smith]

Dennis, this may not be directly relevant, but I've made a backtesting 
app for futures which works reasonably quickly. It uses daily data from 
50 or so markets going back, in some cases, to the mid 70's. If you're 
dealing with intraday data and thousands of stocks then obviously 
that's many, many times as much data, and this will be as much use as a 
sponge leg in a downpour :)

  I tried quite a few approaches, and settled on putting each market 
history into it's own array element and then using revs line and item 
expressions to access the data itself. I also 'retire' data that isn't 
needed anymore, as it steps through, such that rev never has to count 
thousands of crs to get at a particular line.

The app can now step through a twenty year, fifty market array in about 
3 seconds. Sadly, the calculations and storage of running totals etc. 
slow it right down again, and a complete test may take up to twenty 
minutes....still better than the 2 to 3 hours I got on my first 
attempt. I am not a professional or trained programmer, so I can't say 
if I've found the optimal solution for this app, but I think it's as 
optimal as I'm likely to get it.

What I found was that it seemed to be quite slow to populate a large 
and complex array from a file, but quite quick to populate a large and 
simple one. Likewise, slow to find a line or item  thousands forward, 
but quick to to find them tens or hundreds forward.

Probably trying to teach my granny to suck eggs,

Mark