Use GNU parallel (to compress large files)

GNU parallel is a great utility to take advantage of multiple cores. You can download the latest source from the bottom of this page.

The installation is pretty straightforward also. Unzip, ./configure, make, and sudo make install. This is it with GNU software, it just works.

To use this tool, for example zipping a list of files

find `pwd` -name "*.sdf" | sort | parallel -j4 --will-cite bzip2 -9

The -j4 limits the bzip2 process to 4 processes and the –will-cite removes the extremely annoying citation message (something about donating 10,000Eur).