Putting together info from several sources, there are a few options.
The conclusion is that only with command parallel you might get what you want, see below.
Important notes:
- I have made tests with
cp for copying. You should also consider speedups (or down!) obtained with rsync vs cp or other alternative commands, combined with parallel.
- I have tested copying only one file. Results might change if copying many files (e.g., combining a few large files, as you need, with many other small files and subdirectories).
For each of the options, I have tested both
time <option #N, copying to one target>
time <option #N, copying to two targets>
to get a comparison, with a file of 1.2Gb.
Moreover, in some cases I tested two or three times the same command, to assess the dispersion in the results.
I did not compute averages and standard deviations, but the results are obvious.
This is what I got under the testing conditions specified above, with brief comments.
I have concatenated in a single row the results of multiple tests, whenever available.
The base case:
$ time cp -p source/file1 target1/
real 0m0,846s 0m0,680s 0m0,659s
user 0m0,000s 0m0,001s 0m0,016s
sys 0m0,777s 0m0,662s 0m0,643s
The copying options:
Option parallel
$ parallel cp -p source/file1 ::: target1/
real 0m0,745s 0m0,740s
user 0m0,121s 0m0,108s
sys 0m0,609s 0m0,619s
$ parallel cp -p source/file1 ::: target1/ target2/
real 0m0,794s 0m0,860s
user 0m0,116s 0m0,134s
sys 0m1,300s 0m1,380s
Option tee (appending > /dev/null to avoid output to stdout)
$ tee target1/file1 < source/file1 > /dev/null
real 0m0,874s 0m1,040s 0m1,028s
user 0m0,160s 0m0,172s 0m0,137s
sys 0m0,714s 0m0,868s 0m0,887s
$ tee target1/file1 target2/file1 < source/file1 > /dev/null
real 0m1,802s 0m1,680s 0m1,833s
user 0m0,136s 0m0,212s 0m0,197s
sys 0m1,642s 0m1,468s 0m1,619s
Copying to two targets roughly doubles the time for one target, which is slightly larger than the time for the base case.
Option xargs
$ echo target1 | xargs -n 1 cp -p source/file1
real 0m0,666s
user 0m0,021s
sys 0m0,646s
$ echo target1 target2 | xargs -n 1 cp -p source/file1
real 0m1,197s
user 0m0,018s
sys 0m1,173s
Copying to two targets roughly doubles the time for one target, which is similar to the time for the base case.
Option find
$ find target1 -exec cp -p source/file1 {} \;
real 0m2,167s
user 0m0,017s
sys 0m1,627s
$ find target1 target2 -exec cp -p source/file1 {} \;
real 0m3,905s
user 0m0,020s
sys 0m3,185s
Copying to two targets roughly doubles the time for one target, which is much larger than the time for the base case... a clear loser.
Sources for "multiple copying":
- https://www.cyberciti.biz/faq/linux-unix-copy-a-file-to-multiple-directories-using-cp-command/
- How to copy a file to multiple folders using the command line?
- https://stackoverflow.com/questions/195655/how-to-copy-a-file-to-multiple-directories-using-the-gnu-cp-command
Sources for performance cp vs. rsync:
- https://unix.stackexchange.com/questions/91382/rsync-is-very-slow-factor-8-to-10-compared-to-cp-on-copying-files-from-nfs-sha
- https://lwn.net/Articles/400489/
- https://superuser.com/questions/1170636/why-is-there-a-write-speed-difference-between-dd-cp-rsync-and-macos-finder-to
- What's the difference between ` cp ` and ` rsync `?