Allocating more ram to a particular program
^^for encryption/decryption/encoding/compressing/decompressing no. of processor cores matter not ram.
doomgiver
Warframe
if you are using linux, try archiving in the tar.lzma/tar.7z format.
lzma is far faster, but 7z gives slightly better compression (roughly ~10% better, depending on type of file compressed)
i had a whole folder of savegames from many games, total 11+ GB.
tar.lzma took 2 hours 15 minutes, and gave an archive of size 1.1GB.
tar.7z took ~3 hours and gave archive size of 800 GB
tar.bz2 took 2 hours 30 minutes and gave archive size of 1.75 GB
files were savegames, a couple of gameplay videos, pictures (you know, the kind where it shows where the game was saved, like witcher and skyrim, etc) and text files.
it was done on a AMD 955 BE 3.2Ghz with 2 GB DDR3-1333Mhz running Ubuntu 12.10.
imho, the compression was extreme due to the repetitive nature of the savegames.
with "normal" files, the compression would be lesser.
TL;DR : you want better compression in lesser time?
compress in linux. windows tools suck.
lzma is far faster, but 7z gives slightly better compression (roughly ~10% better, depending on type of file compressed)
i had a whole folder of savegames from many games, total 11+ GB.
tar.lzma took 2 hours 15 minutes, and gave an archive of size 1.1GB.
tar.7z took ~3 hours and gave archive size of 800 GB
tar.bz2 took 2 hours 30 minutes and gave archive size of 1.75 GB
files were savegames, a couple of gameplay videos, pictures (you know, the kind where it shows where the game was saved, like witcher and skyrim, etc) and text files.
it was done on a AMD 955 BE 3.2Ghz with 2 GB DDR3-1333Mhz running Ubuntu 12.10.
imho, the compression was extreme due to the repetitive nature of the savegames.
with "normal" files, the compression would be lesser.
TL;DR : you want better compression in lesser time?
compress in linux. windows tools suck.
doomgiver
Warframe
use 7zip (program) and use 7z(archive format) and LZMA (compression method).
or you can choose bzip2 as both archive format and compression method. or mix and match to get desired results.
fir better results, play around with the dictionary size.
you are lucky, you get a gui with 7zip. we have to use a CLI in linux
anyway, here is some more info :
.zip files can be opened on ANY platform, so, if you wanna send someone a compressed file that can be opened on just about any machine, use .zip
.rar files arepretty popular, but they dont compress as much or as fast as 7z files.
also, compression depends on a LOT on the type of files you wanna compress.
lots of pictures? you wont get more than 2-10% compression.
lots of movies? lucky you, movies compress really well.
lots of text files? awesome!! text files compress like nobody's business.
so, match the compression algorithm to your needs, to get the best results.
^^you can not compress much already compressed data like iso,dvd-rips/blu-ray rips,jpeg,exe etc no matter what software/os/hardware you use so for average users compression is useless/insignificant as almost 90% data in their hard disk is not much compressable.
I got excited by the stats posted by doomgiver
Anyway, thanks for the information though
doomgiver
Warframe
as i said, the compression was so good coz the type of data that was to be compressed.
compression works somewhat like this :
data to be compressed :
aaaaaaaabbbbccccccccccdddd
we can write it as :
a8b4c10d4
so, we have compressed the data by 66%
now, if the data was like this :
aaabbbcdeeeeeeeeeeffgghhhh
we write as:
a3b3c1d1e10f2g2h4
we have compressed data by just 34%
IRL, you'd be lucky to have more than 30-40% compression with assorted data files.
if i get the time, i'll do a detailled shootout of the compression techniques.