User Tools

Site Tools


howto:linux_gnu_parallel

Use GNU parallel (to compress large files)

GNU parallel is a great utility to take advantage of multiple cores. You can download the latest source from the bottom of this page.

The installation is pretty straightforward also. Unzip, ./configure, make, and sudo make install. This is it with GNU software, it just works.

To use this tool, for example zipping a list of files

find `pwd` -name "*.sdf" | sort | parallel -j4 --will-cite bzip2 -9

The -j4 limits the bzip2 process to 4 processes and the –will-cite removes the extremely annoying citation message (something about donating 10,000Eur).

howto/linux_gnu_parallel.txt · Last modified: 2018/10/09 10:12 by 127.0.0.1