Score:2

How can I apply labels to a promtail static_config?

tn flag

I am collecting logs using rsyslog from about 5000 servers. My collector is writing all logs to a single file on an NFS volume using RFC5424 format. I am mounting this NFS volume on my promtail nodes, and using static_config to scrape the file. I can view the logs in Loki.

My problem: I don't see any labels in my log entries. I am unable to do LogQL queries based on hostname or any type of query based on facility.

This is the relevant portion of my promtail conf:

scrape_configs:
- job_name: system
  static_configs:
  - targets:
      - localhost
    labels:
      job: varlogs
      __path__: /data/rsyslog.log

Is there any way to apply labels to a static_config?

Score:1
by flag

Yes you can do that with static_config, you will need to use Promtail's pipeline stages to parse the logs and extract the required information to create the labels.

Here could be your modified version:

scrape_configs:
- job_name: system
  pipeline_stages:
    # Assuming the logs are in RFC5424 format, use the regex to extract the hostname and facility
    - regex:
        expression: '.*?<(\d+)>(\d+)\s\d+-\d+-\d+T\d+:\d+:\d+.\d+\S+\s+(\S+)\s+.*'
        output:
          source_labels: [facility, hostname]
    # Map the extracted facility number to a facility name
    - labels:
        facility:
          replacement: "${1}"
    - labels:
        hostname:
          replacement: "${2}"
  static_configs:
  - targets:
      - localhost
    labels:
      job: varlogs
      __path__: /data/rsyslog.log

You will need to adapt the regex to match the specific format of your logs.

I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.