syslog-ng: Timezone change and $HOUR macro

Christopher Causer

6/13/24, 4:16 PM

Here's a log line from an appliance coming in as UTC:

<166>2023-06-13T16:03:44Z ..................

We store this as in files, in Europe/London timezone:

destination d_all {
    file("/srv/logs/$HOST-$FACILITY-$YEAR-$MONTH-$DAY-$HOUR.log"
         time_zone("Europe/London");
};

The time_zone("Europe/London") here works in that the log lines themselves have been converted. However, the $HOUR in the filename is resolutely not shifting from UTC and we end up with logs not being filed into the correct location (they're out by one hour in summer).

How do I update the $HOUR to use the timezone specified in the destination stanza? The documentation seems to say "When macro expansions are used in the destination filenames, the local timezone is used.", but that is not my experience as Europe/London is my local timezone and yet it is choosing the $HOUR of the log file being received.

I am unable to change the time-zone for the appliance. That's just how it is sadly.

This is on a Debian system running 3.28.1-2+deb11u1

1 + 4

syslog-ng

bazsi77

6/14/24, 5:18 PM

The documentation definitely has the intent, and while bugs can happen, I dont remember issues in this part of the codebase. The local timezone is detected based on /etc/localtime. But you can also specify it in the confirmation using the local-time-zone() setting. This can be set in your global options or even locally in the file destination.

Christopher Causer

6/14/24, 6:03 PM

Thanks. By "in the file destination", do you mean using time_zone("Europe/London") as in the config snippet above? That seems to work for the timestamp within each log entry, but the $HOUR macro from which the filename is based is stuck on UTC time. Is there somewhere else that you're referring to? This is one appliance out of many going through the same tcp/udp source. All other appliances seem to send in Europe/London time, so any change will need to work with these other hosts.

Christopher Causer

6/15/24, 1:18 PM

This isn't an answer, as it has problems but changing $HOUR to $C_HOUR uses the hour as on the syslog collector, which is in localtime. This could lead to entries ending up in the wrong file due to the asynchronous nature of what's going on.

bazsi77

6/19/24, 5:50 PM

I meant using the local-time-zone() option, which you can specify in the global options block. That should change the timezone which is used by the filename expansion. Like you said, syslog-ng uses the local time zone for that and you can override its value just like you can do the same with content-related templates.

Score:0

Server

bazsi77

7/6/24, 12:18 PM

To reiterate the points made in the comments, the filename portion of a file destination if it contains macros uses the local timezone as syslog-ng knows it. The default value of the local timezone is detected based on the system settings (e.g. the /etc/localtime file), but can also be overridden within syslog-ng.

To override the local time zone, you can use the local-time-zone() option, which is valid both in the global options block:

options { local-time-zone("EST"); };

or in the file destination declaration:

destination d_all {
    file("/install/log.$HOUR" local-time-zone("EST"));
};

If you otherwise believe that the system's default timezone is properly set, check out what the "date" command reports:

$ date
Thu Jul  6 12:15:34 UTC 2023

This shows that my local time zone points to UTC (this is in a container). If I use my host's date command:

 date
Thu Jul  6 02:16:20 PM CEST 2023

e.g. it is now using CEST. This setting is controlled by /etc/localtime, which is this on my system (Debian):

$ ls -l /etc/localtime
lrwxrwxrwx 1 root root 35 Jun 19 18:58 /etc/localtime -> /usr/share/zoneinfo/Europe/Budapest

Other distros may implement this differently (e.g. copying the file instead of symlinking it), and most distros also have some kind of high level interface for changing this setting.

+ 0

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: syslog-ng: Timezone change and $HOUR macro

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.