Why python-journald-rsyslog seems to batch/dely log write?

moutonjr

2/21/24, 8:33 AM

I intend to log a python script output. Here what it looks like :

arm@stackoverflow > cat test.py
#!/usr/bin/python
from time import sleep
while True:
        print("Hello")
        sleep(1)

I try to use standard syslog way to handle logs, I therefore tried to configure JournalD & SystemD to send everything as Syslog:

arm@stackoverflow > cat /etc/systemd/journald.conf | grep -vP "^#"
[Journal]
ForwardToSyslog=yes

arm@stackoverflow > cat /etc/systemd/system/testservice.service
[Unit]
Description="Test service"
StartLimitIntervalSec=500
StartLimitBurst=10

[Service]
Type=Simple
Restart=on-failure
RestartSec=5s
ExecStart=/home/pi/test.py
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=testservice
   
[Install]
WantedBy=multi-user.target

Finally I use Debian's default Rsyslog to route my logs in some file:

arm@stackoverflow > tail -n1 /etc/rsyslog.conf
if $programname == 'testservice' then /var/log/testservice.log
arm@stackoverflow > sudo touch /var/log/testservice.log && sudo chmod a+rw /var/log

But when I start the whole, all logs seems to be clogged somewhere, and are releases at one hours after start (like a big despool):

arm@stackoverflow > sudo systemctl start testservice.service && sleep 30

Then:

arm@stackoverflow > tail /var/log/testservice.log #nothing

Tests:

arm@stackoverflow > sudo systemctl status testservice.service
● testservice.service - "Test service"
     Loaded: loaded (/etc/systemd/system/testservice.service; disabled; vendor preset: enabled)
     Active: active (running) since Tue 2023-02-21 09:20:06 CET; 1min 33s ago
arm@stackoverflow > ps aux | grep test
root     23488  0.0  0.6  13956  5968 ?        Ss   09:20   0:00 /usr/bin/python /home/pi/test.py

Do you know how to get these logs in realtime?

Misc. This is a test server on Raspberry Pi OS (ARM)

1 + 3

python

systemd

journald

Gerald Schneider

2/21/24, 8:54 AM

Please don't [cross post](https://serverfault.com/questions/1123389/why-python-journald-rsyslog-seems-to-batch-dely-log-write). As this obviously is not in a business environment I'm voting to close as off topic.

meuh

2/21/24, 1:56 PM

As Python to run unbuffered stdout, either run `-u`, or set `PYTHONUNBUFFERED=1` in env.

moutonjr

2/21/24, 7:05 PM

@GeraldSchneider ... Indeed this is for business purposes. I just anonymized/simplified content. If it should be elsewhere, please tell.

Score:0

Server

moutonjr

2/21/24, 7:14 PM

Thanks to @meuh, Adding flag -u to the python script did the trick. I added it to the shebang line:

#!/usr/bin/python -u

+ 0

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Why python-journald-rsyslog seems to batch/dely log write?

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.