Score:0

Read logs real time with bash

be flag

I have this in the system logs:

Feb 27 02:53:51 Latitude-E6430 rsyslogd: action up   7
Feb 27 02:53:51 Latitude-E6430 rsyslogd: action down B
Feb 27 02:53:51 Latitude-E6430 rsyslogd: action up   U
....etc

I want to make a script that is monitoring the logs full time as a service, but I don't know how to do the condition.

1 - That when you read, line by line, store it in a variable.

2 - read variable if it finds in the line, the words [rsyslogd] and [7] it shows [echo "Found" ]

It would be a loop to read the file line by line, looking for that pattern of words.

Example Script

path_full_log="/home/full.log"


function reader_time_real(){

    while read full_log; do
        chek7=$(cat $full_log | grep "rsyslogdd" | grep 7 )
            if [[ $? = 0 ]]; then 
                echo "Found 7";

            fi

        chekB=$(cat $full_log | grep "rsyslogdd" | grep B )
            if [[ $? = 0 ]]; then 
                echo "Found B";

            fi
    done < $path_full_log
}
David avatar
cn flag
Show the script you have made so that someone can see what the errors might be.
BurstBass BurstBass avatar
be flag
done... more down
Soren A avatar
mx flag
Please add your script to your question, not as an answer ... it will probably get closed.
WU-TANG avatar
cn flag
Isn't this a function of rsyslog naturally? I've only configured a syslog-ng server, but I'm thinking they are similar enough that both of them can do filters???
BurstBass BurstBass avatar
be flag
they are not the same, I specify in both that it has to have the word "rsyslogdd" and the 2 possible variants, 7 or B
BurstBass BurstBass avatar
be flag
Fixed, just change `cat $full_log` for `echo $full_log`
Lorenz Keel avatar
gr flag
@BurstBass BurstBass if you fixed it, put an answer by yourself with the final version of the script, then accept your own answer.
WU-TANG avatar
cn flag
@BurstBassBurstBass If your response was to me (you didn't tag anyone), then I think you misunderstood what I was saying.... I was saying that "I think" that syslog already has the built-in capability to do what your script is doing. I've configured a syslog-ng server before and I set up filters to catch words and send them to a separate log (even trigger commands). I would think rsyslog has the same functionality.
Score:1
in flag

Your script is not suitable to do what you want to do. It reads the file only once, but completely. If you loop this, you won't get only new output and it would be a performance nightmare.

Also, you have a useless use of cat, you grep a fixed string with a regex pattern and save the result in a variable you don't use anymore. Also you should not use unquoted filename variables.


Use tail -f | grep --line-buffered:

tail -fn0 "$path_full_log" \
| grep -F --line-buffered 'rsyslogdd' \
| grep -Eo --line-buffered '(7|B)' \
| while read line; do
      echo "Found $line"
  done

Or using xargs:

tail -fn0 "$path_full_log" \
| grep -F --line-buffered 'rsyslogdd' \
| grep -Eo --line-buffered '(7|B)' \
| xargs -I{} echo "Found {}"
  • tail -fn0 reads new lines of the file when they are added.
  • grep -F searches for a fixed string in that line.
  • --line-buffered tells grep to read lines when they come in instead of wait for the end of the input stream which is default.
  • grep -Eo searches for the ERE pattern (-E) (7|B) and outputs only the matched string (-o).
  • while read line or xargs -I{} takes that output and runs your command on the output (here: echo).
Score:1
vn flag

I've made the following script, which updates every second, and highlights any new log entries. It also defines the area in which the log should be displayed (with the logrows and logcols variables), so it's made for use in a fixed size terminal window (I'm using it inside tmux).

You could probably modify this script to highlight only lines that fit a specific criteria, as you indicate.

#!/bin/bash

# Initial configuration variables
logrows=31
logcols=127

# Local function definition
printnew() {
  clear
  echo -e "\e[0;30;47m"
  hightail=$(tail -n "$logrows" /var/log/syslog | cut -b "1-$logcols")
  echo -ne "${hightail}\e[0m"
}

printdif() {
  (( newrows = logrows - logdiff ))
  clear
  normtail=$(tail -n "$logrows" /var/log/syslog | head -n "$newrows" | cut -b "1-$logcols")
  echo -n "$normtail"
  echo -e "\e[0;30;47m"
  hightail=$(tail -n "$logdiff" /var/log/syslog | cut -b "1-$logcols")
  echo -ne "${hightail}\e[0m"
}

printold() {
  clear
  normtail=$(tail -n "$logrows" /var/log/syslog | cut -b "1-$logcols")
  echo -n "$normtail"
}

# Loop every second
while true
do
  sleep 1 &

  # Read log sizes
  logsize=$(wc -l /var/log/syslog | cut -d ' ' -f1)
  (( logdiff = logsize - presize ))

  # If more than $logrows lines are new
  if (( logdiff > "$logrows" ))
  then
    printnew
    redraw=1
  # If less than $logrows lines but more than 0 lines are new
  elif (( logdiff > 0 ))
  then
    printdif
    redraw=1
  # If no lines are new
  else
    if [[ "$redraw" -eq 1 ]]
    then
      printold
      redraw=0
    fi
  fi

  presize="$logsize"
  wait # for sleep
done

The script is made with 3 scenarios in mind:

  1. The entire contents of logrows is new - everything is highlighted (printnew).
  2. Some lines of the log is new - these are highlighted (printdif).
  3. No new log lines - nothing is highlighted (printold).

Disclaimer: The code could probably be optimized, but I'm generally more concerned with my code being readable rather than as condensed as possible.

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.