Re-2: compress <-> revZip

[Phil Davis]

Hi Claus,

It seems like you could compress a very large file by 
reading/compressing/writing one chunk (say 10 MB) at a time, until you 
have compressed the entire file. This approach would use the "open file 
/ read from file / write to file / close file" approach instead of "get 
url / put x after url". That way only 10 MB of the file (in my example) 
would be in memory at any time.

This approach would require that the decompression tool be a custom 
tool. So if you are creating a solution that is Rev from end to end, 
this could work! Otherwise it won't.

Problem: When compressed, each 10 MB chunk will have some unpredictable 
size. You would have to make a way for the decompression tool to know 
the length of each compressed chunk in the file.

One possible solution: Keep track of the compressed sizes as you 
compress each chunk, and write all the sizes (comma-delimited) to the 
output file as a header record when all compression is finished. Then 
the decompression tool would start by reading the header record. The 
numbers in the header tell the decompress tool how much data to grab in 
each read.

Food for thought -
Phil Davis



runrev260805 at m-r-d.de wrote:
> Hi Claus,
>
> tried revzip out here with 4GB data on a 3gig Machine: No, it does not work.
>
> Btw:The same does work here with 7-zip.
>
> Regards,
>
> Matthias
>
> -------- Original Message --------
> Subject: Re: compress <-> revZip (10-Okt-2009 14:03)
> From:    Claus Dreischer <claus at dreischer.de>
> To:      runrev260805 at m-r-d.de
>
>   
>> umm ... noone?
>>
>> How about only the last question:
>>
>> Can i compress/revZip a 2gig file on a machine with 1gig main memory?
>>
>> Regards,
>> 	Claus.
>>
>> Claus schrieb am 24.09.2009:
>>     
>>> Hi,
>>>
>>> can someone please explain me the difference of these two compression
>>> methods in Revolution?
>>>
>>> Especially regarding the filesize which can be compressed/ziped and
>>> the quality of the compression/zip.
>>>
>>> Compress obviously can only compress what fits into the main memory.
>>> What about revZip?
>>> Can revZip compress a 2gig file on a 1gig machine?
>>>
>>>
>>> Regards,
>>> 	Claus.
>>>       

-- 
Phil Davis

PDS Labs
Professional Software Development
http://pdslabs.net