Score:0

Shell script to delete all files of certain type in different folders, except 10 latest

eg flag

First of all, please excuse my poor english.

I am trying to write a shell script to clean up some files of certain type in my backup folders.

I need to remove files of certain type, except 10 newest in each folder.

Folder structure looks like this:

Root folder
│
├── folder_1
│   ├── file_1.txt
│   └── file_2.txt
│   └── file_n.txt
│
├── folder_2
│   ├── file_1.txt
│   └── file_2.txt
│   └── file_n.txt
│
├── folder_n
│   ├── file_1.txt
│   └── file_2.txt
│   └── file_n.txt
│

I've used script from David Foerster as a basis, but can't figure out on how to make it work in different folders separately without manually writing each folder name in script.

Currently, script looks like this:

find /volume1/rootfolder/ -type f -name *.txt -printf '%T@ %p\0' |
sort --zero-terminated --reverse --numeric-sort --field-separator=' ' --key 1,1 |
gawk -F ' ' -v RS='\0' -v ORS='\0' -v retain_count=10 \
  'BEGIN{ maxage = systime() - retain_younger_days * 24 * 3600; }
  (NR > retain_count) && (int($1) < maxage) { print(substr($0, length($1) + 2)); }' |
xargs -r0 -- rm --

But the issue is that it removes all files in all folders, except 10 latest. So, overall i have just 10 files in all folders, not 10 in each folder.

Please, help me figure out, how to make script to process each folder separately.

hr flag
This kind of thing is easier in `zsh`, which has glob qualifiers for selecting files based on modification time. See for example [Bash script: Conditionally delete older files while keeping latest copies](https://askubuntu.com/a/1352544/178692) or [Remove all files within directory older than the most recently added ten files](https://askubuntu.com/a/930709/178692)
ng flag
Sorry @steeldriver I hope you don't mind the question - I was just thinking why not write a Python script? Bash feels problematic to maintain at times for me (in that I agree with extending the scope for the solution)... would it be less adequate in your personal opinion, if so, why?
hr flag
@brezniczky I just meant that it's easier in zsh *than in bash* - I'm not fluent enough in python to comment on its relative merits
ng flag
@steeldriver all right, I'm - almost obviously - unfamiliar with zsh, thanks anyway for now! may someone with both/all 3 cards in hand bump into the question someday and give me/us the answer :)
Score:2
cn flag

You divide it into two parts. First, you build a directory structure, and in this case, we only need directories containing files. In the second part, we sort the newest file first and skip those with tail.

#!/bin/bash

while IFS= read -rd ''; do
    find "$REPLY" \
    -maxdepth 1 -type f -name '*.txt' -printf '%T@\t%p\0' | sort -zk1rn | cut -zf2- | tail -zn +$((10+1)) | xargs -r -0 echo rm
done < <( \
    find $PWD/data \
    -mindepth 2 -type f -printf %h\\0 | sort -zu \
)

There is also the possibility to use globstar with a for-loop:

#!/bin/bash
shopt -s globstar nullglob

for a in data/*/**/; do
    find "$a" \
    -maxdepth 1 -type f -name '*.txt' -printf '%T@\t%p\0' | sort -zk1rn | cut -zf2- | tail -zn +$((10+1)) | xargs -r -0 echo rm
done

Lastly, we make use of an array of counters to keep track of the file count:

#!/bin/bash

find $PWD/data -mindepth 1 -type f -name '*.txt' -printf '%T@\t%p\0' \
    | sort -zk1rn | cut -zf2- \
    | awk -F / -vRS='\0' -vORS='\0' '{a=$0; NF=NF-1} b[$0]++ >= 10 {print a}' | xargs -r -0 echo rm
Nazar Tareyev avatar
eg flag
Thank you for a reply. What if directory depth is more than 1? I simplified my folder structure just to make my question simpler. But real folder structure is deeper and not fixed. Sometimes my target files is under several directories, sometimes more, or less.
bac0n avatar
cn flag
I used the wrong specifier. You can add `-n 1` to `xargs` for better readability.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.