Windows and OSX 64-bit builds?

[Richard Gaskin]

I would second that, with an extension:  it would be most helpful to 
have some guidance on memory management overall.

Many see the total addressable space afforded by 64-bit systems as 
"Cool, I can use a 16-terabyte array!"

But as we've seen in bits and pieces here and elsewhere, total 
addressable memory is not what LC will necessarily use, or can use, for 
many tasks.  And it some cases it may not be what we really want to do.

OS APIs impose their own limits, and many other factors can come into 
play as well.

It would be useful to have at least some general guidance on what those 
limitations are, both in terms of LC as it is today and in terms where 
it bumps into system limits and general memory management challenges.

Often these implications aren't obvious.

For example, this morning I noted the difficulty with exporting an image 
of a 5000 X 5000 px button on Ubuntu.  Later I tried a set of much 
smaller objects but spread out much more broadly (10k px and more), and 
LC remained very responsive throughout - even exported the snapshot fine.

So I'm guessing from this that the difficulty with making unusually 
large standard controls was specific to how Linux' GDK handles them 
internally.  Indeed, making LC-handled objects, like graphics, 
apparently have no problem being rendered and buffered at such sizes.

Rather than make this an attempt at an exhaustive study of memory 
implications (though I'd gladly read it, it would take too long to 
write), it might be nice if Mark could draw from examples he's seen in 
the community over the years to outline some of the more commonly 
requested scenarios.

-- 
  Richard Gaskin


hh wrote:

> Please post this "Split it!"- answer, as it is, in LC's blog.
> This is good even for real beginners.
>
> Large files or large data shouldn't be a reason for _incomplete_ 64Bit
> implementations that would make once again LC Script slower.
>
>> Mark Waddingham wrote:
>> >
>> > Tom Glod wrote:
>> > I will... if u wanna replicate...put an image on a stack..make it
>> > 32k x 32k
>> > ..... and try and do a export snapshot of the image,  LC goes POOF...
>> > Trevor
>> > said tha last version of 8 (8.13)  had some memory issues solves, so i
>> > will try to test is there too.
>>
>> Currently, the engine implements 'export snapshot' by allocating a
>> raster (32-bits per pixel) of the size of the snapshot you are making,
>> rendering into it and then compressing it.
>>
>> So really the maximum size you could hope to snapshot is 16k x* 16k
>> pixels as that requires 2Gb - the engine in general uses signed integer
>> indicies, so the maximum memory buffer it can manipulate is 2Gb bytes. A
>> 32-bit process would probably struggle to do that (due to only having
>> around 2-3Gb of user address space to use) - as there is overhead in
>> rasterization and then compression; but a 64-bit process should be fine.
>>
>> There is a bug here as (at least in this specific case) the engine
>> should fail gracefully (as we know there is a hard limit in the size of
>> an image the engine can process).
>>
>> As you correctly point out 32k x 32k comes in at 1Gb pixels - which at
>> 24-bit RGB comes out at 4Gb of data. No 'normal' 32-bit application
>> which isn't explicitly designed for manipulating huge images will be
>> able to deal with something that size. I would expect applications such
>> as Photoshop to be able to deal with them though since I believe their
>> native raw storage format for images pages from disk as required (so you
>> never have the 'whole thing' in memory at once - just the bit you are
>> looking at / editing).
>>
>> One important thing to remember is that the amount of memory required to
>> take a snapshot is (SOMECONSTANT * 4 * the number of pixels) in the rect
>> of the snapshot (I've not checked but I would estimate 0 < SOMECONSTANT
>> < 2) which means that you can get LiveCode to generate very large
>> images, but you have to break the problem down by splitting up the
>> snapshot into bands and probably use an external (command-line) tool to
>> compress the image into your format of choice (how big an image such a
>> tool can process, again, will be dependent on whether it needs to load
>> the entire image into memory to compress it or not).
>>
>> Rough idea:
>>
>>      repeat with i = 0 to pHeight / kBandSize
>>         import snapshot from rect (0, pWidth, i * kBandSize, kBandSize)
>>         write the imageData of the last image to file tOutputFile
>>         delete the last image
>>      end repeat
>>
>> After this you will have a very large file with the raw xRGB data in it,
>> so you need to find a tool which can take raw 32-bit xRGB data (with
>> specified order of the RGB), the width and height and process it into
>> jpg or png or whatever format you require (I'm hoping others who know
>> more about such things might be able to chime in here - ImageMagick has
>> an arsenal of command-line tools, for example).
>>
>> Warmest Regards,
>>
>> Mark.
>>
>> P.S. There is a hard limit of co-ordinate magnitude in LC and thus the
>> size of any object - 32767 pixels on any side of anything.