Making Revolution faster using dimensioned arrays

[Dennis Brown]

On Jul 5, 2005, at 9:26 AM, Alex Tweedly wrote:

> Dennis wrote:
>
>
>> "These declared arrays would run 10 to 100 times  faster than the  
>> type-less ones for the kind of operations that are  required by  
>> image processing, AI, statistics, and many other  applications."
>>
>>
> Jon wrote:
>
>
>> Given the detailed work that Alex (?) did a week or two ago with  
>> my algorithms, I am not at all sure that your optimism is  
>> warranted.  That said, I love the idea <grin>
>>
>
>
> I too love the idea. I think it would extend the scope of pure- 
> Transcipt to cover a wide range of interesting problems. When it  
> coalesces into a solid Bugzilla enhancement request, it will get  
> quite a few of my votes.
>
> But this on its own will not make pure-Transcript feasible for  
> image processing tasks.
>
> Imagine that the engine could be incredibly fast at accessing  
> these"dimensioned arrays". We can easily benchmark to see how long  
> it would take for various assumptions about how fast it *could*  
> possibly be.
>
> For a 6MP loop:
> doing nothing take 430 ms
> assigning to a scalar variable takes 1500 ms
> accessing via an index (char i of var) takes 2500 ms
> incrementing via an index (char i of var) takes 5200 ms
> idealized version of histogram takes 10400 ms
>
> So even if the optimization of "dimensioned arrays" achieves  
> everything it can hope for, and can access those arrays through a  
> single index look-up - it will still take 10 seconds to do a  
> histogram for an image of the size produced by last year's consumer  
> cameras.
>
> It is still, IMO, a worthwhile addition to the language, and would  
> allow us to do a lot of things that are beyond pure-Transcript  
> today but are less demanding than image processing. In some future  
> combination with a "compileIt" optimization (and perhaps some  
> optimizing of "repeat with" loops) it might stretch to cover image  
> processing.
>
> But honestly, image processing is a specialized area, and I think  
> it might well be better served by using a specialized (already  
> written) package, such as ImageMagick, or PIL, or .... either  
> through an external, a command-line or process call, or via RPC- 
> like socket communication.
>
> In fact, an RPC-like mechanism would also help with the ability to  
> take full advantage of multi-core or multi-CPU systems. Since the  
> near-term (5-10 year) growth in PC performance is likely to come  
> from multiple processors rather than faster linear CPU cycle times,  
> anyone doing image processing needs to be looking at using multiple  
> threads or multiple processes to gain those benefits - and so an  
> RPC-like mechanism has a lot going for it ...

Alex, Jon,

I completely agree that this would not make Transcript into a  
PhotoShop application.  The image processing I was referring to was  
more along the lines of image manipulation that could be produced 5  
years ago by a "C" program.  In other words, Transcript is lagging  
good compiled languages by two orders of magnitude in array  
processing (not including dedicated vector processors).  That just  
means that we have to look backwards in time to what could be done  
when CPUs ran at 20 MHz.  I would like to see us get up to the 200  
MHz era for general stuff.  Specialized native commands like a  
package of array operators that approximate the capabilities of APL  
would of course improve the performance significantly, but for now I  
would just like to get over the hump of having the base functionality  
that could support better things in the future.

Having this fixed format might also make it possible to have an  
efficient way to call an external with a pointer to the array to add  
a lot of missing functionality --but this is outside my experience.

Dennis