Score:0

How to truncate the log file size without losing recent log files?

ng flag

I want to reduce the log file size to the particular minimum size

I tried the truncate command as below

$ truncate -s 20M log filename

But when I check the truncated file it truncates the recent logs and keeps the old logs

When analyzing further I found the tail and head commands. But the problem is when I use the head command is working and when I use tail command it saves the content in the human not readable format like @@@@@@@@@@@.

$ head -c 10M logfile > newfile (it saves the head of the file as mentioned size)

$ tail -c 10M logfile > newfile(It saves the content which cannot be readable)

Now I want to truncate the large file which removes only starting of the file but not the ending of the file and if possible it should be in one command without creating another file as head and tail Can anyone guide me on this? Thank you!

Marcel avatar
gb flag
This is how I do it: `cat logfile | gzip -c > logfile.gz; :> logfile`
Score:0
hr flag

The approach: tail -c 10M logfile > newfile should work. Try 'which tail' and 'where tail' to see what is wrong.

I have been using tail -c for many years. Since this is getting a bigger thing for me now, I just now wrote a little tcsh script to do it:

#!/bin/tcsh -f
set echo_style='none' ; setenv LC_ALL C #; set echo verbose

# Purpose: reduce the size of log file FILE to SIZE, if it exceeds
#          twice that amount, by junking the beginning of it.

# Usage: reducelog SIZE FILE
# Clobbers FILE.tmp

# check arguments
set msg="Usage: reducelog SIZE FILE"
if ($#argv != 2) goto err
set size="$1"
set msg="Invalid reduced SIZE"
if (`echo "$size" | grep -c '^[1-9][0-9]*$'` != 1) goto err
set file="$2"
set msg="File not accessible: $file"
if !(-frw "$file") goto err

# check whether we should cut
set msg="Error finding the size of $file"
set oldsize="`ls -l "$file"  | cut -d' ' -f 5`"
if ($status) goto err
if (`echo "$oldsize" | grep -c '^[0-9][0-9]*$'` != 1) goto err
@ maxsize = $size + $size
if ("$oldsize" < "$maxsize") goto end

# cut it down quickly (so that hopefully nothing else interferes)
set msg="Error reducing the size of $file"
tail -c "$size" "$file" >! "$file.tmp"
if ($status) goto err
set msg="Error replacing $file"
mv -f "$file.tmp" "$file"
if ($status) goto err

# check that we did reduce the size
set msg="Error finding the new size of $file"
set newsize="`ls -l "$file"  | cut -s -d' ' -f 5`"
if ($status) goto err
if (`echo "$newsize" | grep -c '^[0-9][0-9]*$'` != 1) goto err
set msg="Failed to reduce the file size"
if !("$newsize" < "$oldsize") goto err

# done
goto end

# error
err:
echo "*** $msg\!"
exit 1

# end
end:

The reason I am still googling this is that I am somewhat concerned about the (remote) possibility that some process writes to the log file just as I am moving it into position. But I guess there is simply no other way to find on the web.

Note: in the script above, SIZE is in bytes, e.g. 100000 for 100 kB.

jm flag
If the process writing to the log file keeps the file open, then deleting the original file and replacing it with a new file will not accomplish anything. The open file descriptor will still be writing to the now-deleted file and the file created by the tail command will never be touched.
LeoninT avatar
hr flag
@doneal24: Yes, if the process writing to the log file keeps the file open, the above solution does not work by itself. But read the original question: the poster *wanted* to use tail, (but got some other problem), so this obviously did not apply to him (or me, so far). But I found that fetchmail (in demon mode) _does do this_. So I had to make an adjustment to my script in this case to touch $HOME/.fetchmailrc immediately after moving the new log into position. This causes fetchmail to restart and reopen the log. Thanks for reminding me to add that here.
Score:0
pa flag

Being a fan of simplicity and high maintainability, I made this. I'm open to your opinion.

This command will truncate log file /tmp/myfile, it cuts the beginning and not the end of the file, it does not require you to configure services and install multi-line scripts; just a one-line command you can add to your crontab.

tail -c 1M /tmp/myfile > /tmp/dzqxH4ZMiSQb91uMMMgPhsgmpnc && rm /tmp/myfile && mv /tmp/dzqxH4ZMiSQb91uMMMgPhsgmpnc /tmp/myfile

First command cuts 1MB of the target file into a temp file, second command deletes the target file, and the third one renames the 1MB temp file as your target file.

It has the disadvantage that it will allocate in your file system, temporarily, 1 MB of space. This could be a problem if you want to keep, say, the last 1 GB of your log file, and your storage is too small or too slow.

If that's your case, you could try this:

fallocate -c -o 0 -l 1M file

It will cut off bytes from your file; starting at position 0, it deallocates 1 MB of disk space off your file. I understand it does it in place, instead of copying data into a temporary file. But it takes more complexity; you would need to first calculate the current size of the file to know how much to deallocate, or make a little loop, something like:

while (file larger than 100M) fallocate 1M; 

resulting in a file between 100 MB and 99 MB. But I like simplicity so for now I'm going with the first option.

Andrew Henle avatar
ph flag
`rm /tmp/myfile`??? That means any process with that file open as its log file will continue writing to the file after it's been deleted.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.