trace kubernetes users activities in pods

Michael Cab

7/22/24, 2:33 PM

I want to track users activities in a k8s cluster. for example I want to get k8s username of a user that executes a command in a pod. there is a tool named Tetragon. it can uses k8s api. following log is a sample output from Tetragon on a k8s cluster:

    {
  "process_exec": {
    "process": {
      "exec_id": "bWluaWt1YmU6NzAyMDQ2ODIyNTEwNDg6MTE5MDk1MQ==",
      "pid": 1190951,
      "uid": 0,
      "cwd": "/",
      "binary": "/bin/sh",
      "arguments": "-c \"sleep 60m\"",
      "flags": "execve rootcwd clone",
      "start_time": "2023-07-22T13:37:01.382355175Z",
      "auid": 4294967295,
      "pod": {
        "namespace": "default",
        "name": "alpine-59dcb54bd-l7dtv",
        "container": {
          "id": "docker://767327faa6bc703188e434b74e80ed29f14973556b4411060674056cf9b305d6",
          "name": "alpine",
          "image": {
            "id": "docker-pullable://alpine@sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1",
            "name": "alpine:latest"
          },
          "start_time": "2023-07-22T13:37:01Z",
          "pid": 1
        },
        "pod_labels": {
          "pod-template-hash": "59dcb54bd",
          "run": "alpine"
        }
      },
      "docker": "767327faa6bc703188e434b74e80ed2",
      "parent_exec_id": "bWluaWt1YmU6NzAyMDQ1NTgzNDYxOTA6MTE5MDkzMQ==",
      "refcnt": 1,
      "tid": 1190951
    },
    "parent": {
      "exec_id": "bWluaWt1YmU6NzAyMDQ1NTgzNDYxOTA6MTE5MDkzMQ==",
      "pid": 1190931,
      "uid": 0,
      "cwd": "/run/containerd/io.containerd.runtime.v2.task/moby/767327faa6bc703188e434b74e80ed29f14973556b4411060674056cf9b305d6",
      "binary": "/usr/bin/containerd-shim-runc-v2",
      "arguments": "-namespace moby -id 767327faa6bc703188e434b74e80ed29f14973556b4411060674056cf9b305d6 -address /run/containerd/containerd.sock",
      "flags": "execve clone",
      "start_time": "2023-07-22T13:37:01.258449828Z",
      "auid": 4294967295,
      "parent_exec_id": "bWluaWt1YmU6NzAyMDQ1NTAyNzk4MDM6MTE5MDkyNA==",
      "tid": 1190931
    }
  },
  "node_name": "minikube",
  "time": "2023-07-22T13:37:01.382358135Z"
}

Now I want to have k8s username of the user executed this commands. for example I want to have a field in a above json like:

username: minikube-user

does anyone have an idea?

1 + 0

audit

logging

kubernetes

cilium

Score:0

Server

Mahé

7/31/24, 8:19 AM

Essentially what you need to do is to correlate the k8s audit logs with the Tetragon logs about execution. Using the k8s metadata Tetragon adds to events (pod names, container ID, etc.) and the date you should be able to retrieve who performed the action at the k8s user level.

The part in the event you showed in example:

"pod": {
  "namespace": "default",
  "name": "alpine-59dcb54bd-l7dtv",
  "container": {
    "id": "docker://767327faa6bc703188e434b74e80ed29f14973556b4411060674056cf9b305d6",
    "name": "alpine",
    "image": {
      "id": "docker-pullable://alpine@sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1",
      "name": "alpine:latest"
    },
    "start_time": "2023-07-22T13:37:01Z",
    "pid": 1
  },
  "pod_labels": {
    "pod-template-hash": "59dcb54bd",
    "run": "alpine"
}

Should give you enough context to browse and correlate the k8s audit logs.

+ 0

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: trace kubernetes users activities in pods

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.