RANT (Mild): revZIP library

[Mark Clark]

Have you considered chunking the problem up and using compress v revZip library? I ran into something similar when I wanted to keep memory down for encrypting/decrypting multi-gig log files…got a lot of advice about just using shell—but for the same sorts of reasons as you I wanted to do things within lc. I used a standard 1MB chunk size so just added a visual working on '1 of n progress bar’. That might be more effort than it’s worth to you.

Might be simpler to create 1 or more archives using the revZip library vs rolling your own. You’d need to stat the files to determine the sizes and # of archives and limit each to < 2GB.

the detailed files (synonymous with the long files)


Mark










> On Oct 11, 2022, at 4:14 PM, Paul Dupuis via use-livecode <use-livecode at lists.runrev.com <mailto:use-livecode at lists.runrev.com>> wrote:
> 
> Thank you for the  history.
> 
> Yes, using other ZIP tools via shell is a possibility, but one I would rather not do. While some researchers who use our app are very computer savvy, most are not. Having a LiveCode ZIP in a window with a nice UI that uses terminology our data analysis tool already uses to let them 'package' their data works well for usage and adoption.
> 
> We're already doing invisible shell work-arounds for a number of engine bugs, with using ffmpg to replace Export Snapshot (see https://quality.livecode.com/show_bug.cgi?id=23694 <https://quality.livecode.com/show_bug.cgi?id=23694>) the latest. I would just rather not add another 'work-around'. I was actually spending time today cleaning out an old work-around from when a minimized stack set one of its coordinates to -32000 as a way of indicating it was minimized (before the iconic property existed) and work-around fro when a stack was minimized it would get a 'resizeStack' message with a new width and height of 0,0. None of these apply any more but we don't always have time or awareness to remove work-arounds when they are fixed, which can sometimes be years after the work-around was put into place.
> 
> Anyway, thank you for the suggested approach. It is actually really goo to know someone else has already done it.
> 
> 
> On 10/11/2022 4:34 PM, Pi Digital via use-livecode wrote:
>> Hi Paul
>> 
>> I got affected by that and other win revzip issues too. I ended up parsing them through a shell script and using windows built in unzipper. Of course the downfall with that is a lack of feedback on progress which, for large files like this, requires a great deal of time to process and feedback is almost a necessity. I made the shell visible just so that the customer was aware of its progress. It was ugly in terms of UIX but it got us out of a short term hole. I hope you can get a more long term solution.
>> 
>> I’m sure you’re capable of building the shell script but let us know if not.
>> 
>> All the best
>> Sean
>> 
>> 
>> 
>>> On 11 Oct 2022, at 20:37, Paul Dupuis via use-livecode <use-livecode at lists.runrev.com <mailto:use-livecode at lists.runrev.com>> wrote:
>>> 
>>> We just got bit by a 4+ year old bug (See https://quality.livecode.com/show_bug.cgi?id=20859 <https://quality.livecode.com/show_bug.cgi?id=20859>) where the revZIP library on macOS can create and read ZIP archives over 2GB, BUT the Windows library can not.
>>> 
>>> We distribute our desktop research application as "cross-platform" for macOS and Windows and just had a researcher on macOS build a "package" (our term for a custom ZIP archive our LiveCode app can make of the researcher's data files) over 2GB and sent to their Window colleague who can NOT open it.
>>> 
>>> Yes, we walked the Windows researcher through unzipping the archive with WinZIP and accessing all the data files manually, but so much for the "elegance" of our app being cross-platform due to using LiveCode.
>>> 
>>> Now, do we add code to check the size and prohibit macOS users from creating "packages" over 2GB so if they are sent to a Windows researcher, they can be opened OR do we just warn the Windows researcher if the archive is over 2GB that they will not be able to open it? Neither is a good option.
>>> 
>>> This just makes us look bad.
>>> 
>>> Paul Dupuis
>>> Researchware
>>> 
>>> _______________________________________________
>>> use-livecode mailing list
>>> use-livecode at lists.runrev.com <mailto:use-livecode at lists.runrev.com>
>>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>>> http://lists.runrev.com/mailman/listinfo/use-livecode <http://lists.runrev.com/mailman/listinfo/use-livecode>
>> _______________________________________________
>> use-livecode mailing list
>> use-livecode at lists.runrev.com <mailto:use-livecode at lists.runrev.com>
>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode <http://lists.runrev.com/mailman/listinfo/use-livecode>
> 
> 
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com <mailto:use-livecode at lists.runrev.com>
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode <http://lists.runrev.com/mailman/listinfo/use-livecode>