[Linux4christians] Question: Working With Compressed Files in Brash

[Chris Debenham]

On 24 April 2012 09:47, Don Parris <parrisdc at gmail.com> wrote:

> On Mon, Apr 23, 2012 at 18:34, Chris Debenham <chris at adebenham.com> wrote:
>
>> On 24 April 2012 05:14, Don Parris <parrisdc at gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> <SNIP>
>>>
>>> Anyway, just curious as to whether anyone else sees something I should
>>> think about.  I am considering combining tar and bzip.  Tar will let one
>>> view a list of files in an archive - something that will be useful when the
>>> user wants to see what is in the trash.
>>>
>>> Thoughts?
>>> Don
>>>
>>
>> I would think that compressing each file individually is best/safest.
>> Adding files to a compressed archive is slow - and removing one file from
>> it is even worse (ie when restoring a single file from the trash)
>> If you used tar+bzip2 (or similar) then each operation requires
>> uncompressing the entire archive first.
>> This would be made even worse if you delete a few large binary files that
>> don't compress well (ie: you delete 2 * 1G video files which take up 1.9G
>> total compressed.  When you go to undelete one of them for a short while
>> you will have the original 1.9G compressed archive + a 2G tarball + the 1G
>> file being restored. If you use pipes then the 2G tarball may sit in
>> memory/swap but it will still use a fair bit of memory)
>> Finally due to the way tar works reading the archive to find out what is
>> in it requires a full decompress of the file - which could be slow due to
>> IO, CPU and memory usage
>> If you keep to just using bzip2 (or 7z or gz or lz ;) ) on each
>> individual file then figuring out what is in trash becomes a much easier
>> "ls | sed -e 's/.bz2$//g' " :)
>>
>>
> Hi Chris,
>
> I like that suggestion.   I was curious as to the effect of having a bunch
> of individually compressed files, as compared to sets of files, but I can't
> imagine there would be that much difference.  Your suggestion certainly
> makes viewing the trash contents a lot easier, while still offering some
> space savings.  It also strikes me as easier to merge into my current code,
> as well - I should be able to implement the compression for each file as I
> am looping through them:
>
> log it
> compress it
> move it
>
> Maybe I could establish a $targetfile and set the log as:
> /home/user/dir1/file1 >> trash/file1.bz 20120423_1655
>
> If the user tosses a non-empty directory into the trash, I would need to
> compress the files it contains, removing the original, and then move the
> whole directory.  Still, it should be do-able and not entirely unreasonable.
>

Just as a little word of warning - don't just run 'bzip2 file' or similar
because that fails if the file is already bzip'ed or if compressing would
result in it being larger than the original file.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.thelinuxlink.net/pipermail/linux4christians/attachments/20120424/a51ddd0e/attachment-0001.html>