Score:1

Why can't I attach custom fields to the journal with systemd-run --user?

jp flag

I want to run a program in a transient unit with systemd-run, using the user-level service manager (--user), with some custom fields attached to the journal.

According to the docs I can use LogExtraFields to do this. So I am running the program and setting the custom field FOO=bar, like this:

$ systemd-run --user --property LogExtraFields="FOO=bar" sh -c "sleep 1 && echo henlo"

But when I check the journal entries for the transient unit, my custom label isn't there:

$ journalctl --user-unit run-rd8f0b861d11d4b13bcdae84e5a9d55cf.service -o verbose | grep FOO
$

However, when I try the same operation as root and use the system-level service manager, my custom field is indeed written to the journal:

$ sudo systemd-run --property LogExtraFields="FOO=bar" \
  sh -c "sleep 1 && echo henlo"
$ sudo journalctl --unit run-r75220e0f4e574ba292eb864c65d7f414.service -o verbose | grep FOO
     FOO=bar

Why is this the case? Do users need extra permissions to write extra fields into the journal?

Score:1
fr flag

Why is this the case? Do users need extra permissions to write extra fields into the journal?

Users don't, but the specific mechanism that systemd uses to supply the extra fields to journald is, currently at least, only aware of system-level units (and relies on write access to /run/systemd).

Although your user-level systemd manager could supply extra fields to journald directly, when creating the stdout-to-journal pipe for this service, this would only have effect on messages captured from stdout and stderr – but the fields couldn't be automatically attached if the service process itself submitted messages through /dev/log or through the "main" journald logging socket.

To solve that problem, systemd has chosen to have journald manually query systemd for the extra fields attached to the unit (as journald already knows1 which unit the process belongs to) – but as currently implemented, only the system-wide service manager (pid1) is being queried for fields attached to [email protected]; your user-level services are ignored.

1 (With short-lived processes such as echo there's also the problem of the process exiting before journald can find out which unit it belongs to. Unlike e.g. the PID or UID, which are attached by the kernel to each socket message, the kernel developers are steadfastly refusing to attach cgroup membership data so journald needs to get it from /proc/<pid> after the fact. Which is why you might see _SYSTEMD_UNIT= missing from some messages.)

User processes can include custom fields (except privileged _* fields) if they submit messages directly through the main journald socket. (This is already done e.g. by GLib2 logging functions.) For example:

logger --journald <<EOF
MESSAGE=Hello!
FOO=bar
EOF

(Ignore the --sd-param option if you find it in the manual; the "sd" does not refer to systemd – it's for structured data in the RFC 5424 "IETF Syslog" protocol.)

Tom Donohue avatar
jp flag
What a detailed answer, thank you. Is that behaviour (that journald queries systemd for fields) documented somewhere?
user1686 avatar
fr flag
I don't think so. It's an implementation detail, not an API.
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.