Score:0

Ubuntu

How do I move all files that are not copies in two directories into a third?

Michael Stachowsky

5/25/24, 5:25 PM

So, I have two directories, both of which have no subdirectories and are filled only with text files. In one directory, "total", all of the text files are present. In a second directory, "processed", only a small subset of those in "total" are present. I need a third directory, "left_to_process", that contains those files in total not in processed. How can I do that? Unfortunately, we're talking about thousands of files so doing this by hand is not recommended...

3 + 1

bash

Michael Stachowsky

5/25/24, 5:57 PM

I promise it isn't, or are you suggesting I should just google it?

Score:3

Ubuntu

waltinator

5/25/24, 6:28 PM

It's easy, simply (assuming no blanks or funnys in filenames):

cd total
for i in * ; do 
  if [[ \! -f ../processed/$i ]] ; then
     echo "$i"
  fi
done | \
  xargs -r mv -t ../left_to_process

You'll probably have to fix the directories.

+ 1

steeldriver

5/25/24, 7:16 PM

The only "funny" that would cause problems is the newline character I think - I'd probably do something more like `for f in total/*; do [[ -f processed/${f##*/} ]] || printf '%s\0' "$f"; done | xargs -r0 cp -t left_to_process/` which would handle even that edge case

Score:1

Ubuntu

Lance Gerhadsen

5/25/24, 6:42 PM

Here's how I would do it...

Because my understanding is that you want to calculate:

A (total directory files) - B (processed directory files) = C (left to process files)

You could create a new directory, C, and then symbolically link all the files from A into C. Next, for each file in B, execute a rm command on the links in C. What's left will be the files you need to process.

E.g. (using A, B, C as defined above)

mkdir C
cd C
ln -s ../A/* ./
for i in ../B/*
do rm `basename "$i"`
done

Just ls the C directory to see what's left.

+ 0

Score:1

Ubuntu

user1700112

5/28/24, 11:25 PM

This should work better:


#!/bin/bash
#
# Collate directory of files to parcel out those already processed.
#
#
Wild_Card="*"

  # Iterate over the wildcard set.
  cd total
  for fname in $Wild_Card
  do
    # Ensure we have at least one matching file, as in we did not get back the wildcard string.
    if [ "$fname" == "$Wild_Card" ]
    then
      echo No files found.
      exit 0
    fi

    # If this file name is not processed, make a copy in the left_to_process directory.
    if [ ! -f ../processed/"$fname" ]
    then
      cp ./"$fname" ../left_to_process/"$fname"
      echo "$fname"
    fi
done

+ 0

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: How do I move all files that are not copies in two directories into a third?

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.