Score:0

Moving files from 1 large drive to 2 smaller drives

th flag

I'm trying to write a script that will move files from one large drive to two smaller drives - I thought it would be better if it did it alternately. So move 1st file to drive A, 2nd file to drive B, 3rd file to A etc.... It doesn't actually matter that they are done in order, they could be picked at random (I just thought it would be easier) - it's more important that I have 2 drives containing a similar amount of files. or is there a command to just copy ??TB from one drive to another?? or copy until the drive is full.

OS is Ubuntu 18.04.6 LTS

This is the original part of the script that is causing the problem. It starts off copying data to Drive 1 and Drive 2, but then just copies to drive 1. So I end up with one full drive and the other is nearly empty.

DATA=/home/data
DATADRIVES=(/media/Drive1/data /media/Drive2/data)

getLowest () {
    LOWEST=${DATADRIVES[0]}
    for DRIVE in ${DATADRIVES[@]}; do
        if [[ $(du $DRIVE) < $(du $LOWEST) ]]; then
            LOWEST=$DRIVE
        fi
    done
}


echo "Copying data from Drives"
echo "DRIVES: ${DATADRIVES[@]}"
FILESTOCOPY=$(find "$DATA"/* -type f)
for F in ${FILESTOCOPY[@]}; do
    getLowest
    echo "Now copying to $LOWEST"
    rsync --progress -S $F $LOWEST
done
in flag
This sounds like an academic exercise. Could you [edit] your question to include: (0) the version of Ubuntu you're running (1) any script that you've already written to try and perform this task (2) a summary of what's not working with that script. By doing this, it may be possible to offer a solution *and* learn something new along the way
waltinator avatar
it flag
A simpler way would be to copy to A until it's full, copy to B.
bac0n avatar
cn flag
Are these organized in directories?
th flag
@waltinator this does sound like the simplest solution - can you give me an example please of how to do it?
th flag
@bac0n the original data is just in one folder and is to be moved to another drive into a folder of the same name
th flag
@bac0n I can't rename the data unless there is a way to change it back when it's moved - but I think that will overcomplicate it.
th flag
Is there a way I can just say copy 2TB of data from drive A to drive B??
Score:1
cn flag

If you divide your tasks into two steps, first, you divide and save the filenames of the bigger drive into two files. To alternate between the different files, a simple modulus is used which gives us 0, 1, 0, 1 the number is used to resolve the file used for the filenames, .e.g, 0 resolves names_0.txt and so on. The file list should only contain all end nodes (files, links, and empty folders) and no starting point. For the last part cpio is used instead of moving the files (I believe copying does not put as much strain on the heart as mv does). The same principle is used to resolve the targets for cpio.

#!/bin/bash

a=0
b=$PWD
c=$b/source

declare -A A=( \
    [0]=$b/names_0.txt [1]=$b/names_1.txt [$b/names_0.txt]=$b/mnt/target_0 [$b/names_1.txt]=$b/mnt/target_1 \
)

rm -f "${A[0]}" "${A[1]}"
                           #            0,1,0,1
while IFS= read -rd ''; do #            ⌄
    printf %s\\0 "$REPLY" >> "${A[$((++a % 2))]}"
                           #  ⌃
                           #  $b/names_0.txt, $b/names_1.txt
done < <(find "$c" \( \( -type d -empty \) -o ! -type d \) -printf %P\\0 | sort -zV)

#   $b/names_0.txt, ...
#   ⌄
for d in "${A[0]}" "${A[1]}"; do
#                          $b/mnt/target_0
#                          ⌄
    cpio -0 -pvdm -D "$c" "${A[$d]}" < "$d"
done #                                  ⌃
     #                                  $b/names_0.txt

I used a simple install script to generate a test example:

#!/bin/bash

a=0
for i in source/{a..z}/{0..20}; do
    install -D /dev/null $i/file_$((++a)).txt
done
mkdir -p mnt/target_{0..1}
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.