Score:0

rsyslog is running, but nothing is output

de flag

nice to meet you.

Suddenly nothing is being output in /var/log/messages, cron, secure, etc... I have not restarted rsyslog or modified /etc/rsyslog.conf, so I do not know why. Even after rebooting, the output is still not output. There are also servers that are outputting normally.

If you look at /proc/*/fd, the one that outputs correctly is

l-wx------ 1 root root 64 Apr 16  2021 8 -> /var/log/secure
l-wx------ 1 root root 64 Apr 16  2021 7 -> /var/log/maillog
l-wx------ 1 root root 64 Apr 16  2021 6 -> /var/log/cron
lrwx------ 1 root root 64 Apr 16  2021 5 -> socket:[8296]
lrwx------ 1 root root 64 Apr 16  2021 4 -> socket:[8295]
lr-x------ 1 root root 64 Apr 16  2021 3 -> /proc/kmsg
l-wx------ 1 root root 64 Apr 16  2021 2 -> /var/log/messages
lrwx------ 1 root root 64 Apr 16  2021 1 -> [eventpoll]
lrwx------ 1 root root 64 Apr 16  2021 0 -> socket:[8297]

The one that is not output is

lr-x------ 1 root root 64 May  2 05:24 3 -> /proc/kmsg
lrwx------ 1 root root 64 May  2 05:24 1 -> socket:[122587394]
lrwx------ 1 root root 64 May  2 05:24 0 -> socket:[122587391]

The rsyslog.conf is the same for both servers.

# rsyslog v5 configuration file

# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html

#### MODULES ####

$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imklog   # provides kernel logging support (previously done by rklogd)
#$ModLoad immark  # provides --MARK-- message capability

# Provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514

# Provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514


#### GLOBAL DIRECTIVES ####

# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf


#### RULES ####

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                  -/var/log/maillog


# Log cron stuff
cron.*                                                  /var/log/cron

# Everybody gets emergency messages
*.emerg                                                 *

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          /var/log/spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log


# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$WorkDirectory /var/lib/rsyslog # where to place spool files
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList   # run asynchronously
#$ActionResumeRetryCount -1    # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514

We are unclear as to why the output has suddenly stopped.

OS: CentOS6 rsyslog: 5.8.10

Peter Zhabin avatar
cn flag
All your actual rsyslog configuration is in /etc/rsyslog.d/*.conf files. Look in there and specifically look for `& stop` statements, anything that happens to be below this statement (or to be included after the file with this statement) would be unreachable. It is likely that some updates introduced this change for some specific config file in there and now your main configuration isn't reachable anymore.
de flag
Thank you. However, I checked under /etc/rsyslog.d/ and there are no files that have been recently modified.
eDonkey avatar
sl flag
The most common thing if it suddenly "stops working" is, that you have **no disk space** left. But it could also be an **ownership** problem. Rsyslog starts running as **root**, but after some time it drops privileges and runs as **syslog**, which may not have the rights needed (See: [this answer](https://serverfault.com/a/527088/961526)).
de flag
This issue has been resolved. The reason was that there was a setting in a file under /etc/rsyslog.d to forward to another server, but The reason seemed to be that there was a setting in the file /etc/rsyslog.d to forward the logs to another server, but the destination became unknown because the target server was stopped. By deleting this configuration file and restarting the server, the logs are now being output.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.