Score:0

Transparently proxying to nodeport in kubernetes

in flag

I have a kubernetes set up with a pod containing the following containers:

  1. Squid container
  2. transocks (like redsocks); a transparent SOCKS proxy

I am running this in k3s locally on a linux PC and want to transparently proxy all outgoing traffic from the PC through this transparent proxy. So outgoing host traffic forced to the transocks port on the kubernetes pod. Right now it is not working, the connection just times out.

I know the iptables rules I am using are good because they worked when I ran the above containers directly in docker using host networking. squid and transocks users were uid 31 and 32. I created the following iptables rules:

#iptables -t nat -A OUTPUT -m owner --uid-owner 31 -j ACCEPT
#iptables -t nat -A OUTPUT -m owner --uid-owner 32 -j ACCEPT
#iptables -t nat -A OUTPUT -p tcp --dport 80 -j REDIRECT --to-port 12345
#iptables -t nat -A OUTPUT -p tcp --dport 443 -j REDIRECT --to-port 12345

Where 12345 is the port transocks is listening on. Again, these rules work great under docker using host networking. transocks forwards to 127.0.0.1:3128 (squid port) and it is proxied transparently. However, when I try it using a kubernetes pod using a nodeport (30345 for transocks, 30128 for squid), it doesn't seem to be working. The connection just hangs until it times out.

I should mention that I have also tried the same with dns, where I redirect output traffic bound for 53 to port 9053 (dns port mapped in docker), and it works, but when I use it in kubernetes and redirect to the nodeport (30053) it hangs the same way. So for some reason node ports don't seem to work when you redirect traffic to them. I was wondering if someone could tell me what I am missing here. I must not understand kubernetes networking very well.

In case someone wants to see my kuebrnetes manifest, here they are.

Deployment

{
  "apiVersion": "apps/v1",
  "kind": "Deployment",
  "metadata": {
    "name": "webfilter",
    "labels": {
      "app": "webfilter"
    }
  },
  "spec": {
    "replicas": 1,
    "selector": {
      "matchLabels": {
        "app": "webfilter"
      }
    },
    "template": {
      "metadata": {
        "labels": {
          "app": "webfilter"
        }
      },
      "spec": {
        "containers": [
          {
            "name": "squid",
            "image": "jusschwa/squid-ssl:latest",
            "ports": [
              {
                "containerPort": 3128
              }
            ]
          }, {
            "name": "transocks",
            "image": "jusschwa/transocks-proxy:latest",
            "ports": [
              {
                "containerPort": 12345
              }
            ]
          }
        ]
      }
    }
  }
}

Service

{
  "apiVersion": "v1",
  "kind": "Service",
  "metadata": {
    "name": "webfilter",
    "labels": {
      "app": "webfilter"
    }
  },
  "spec": {
    "type": "NodePort",
    "selector": {
      "app": "webfilter"
    },
    "ports": [
      {
        "name": "squid",
        "protocol": "TCP",
        "port": 3128,
        "targetPort": 3128,
        "nodePort": 30128
      },
      {
        "name": "transocks",
        "protocol": "TCP",
        "port": 12345,
        "targetPort": 12345,
        "nodePort": 30345
      }
    ]
  }
}

iptables rules

iptables -t nat -A OUTPUT -m owner --uid-owner root -j ACCEPT
iptables -t nat -A OUTPUT -m owner --uid-owner 31 -j ACCEPT # squid user
iptables -t nat -A OUTPUT -m owner --uid-owner 32 -j ACCEPT # transocks user
iptables -t nat -A OUTPUT -m owner --uid-owner 32 -j ACCEPT # unbound user
iptables -t nat -A OUTPUT -p tcp --dport 80 -j REDIRECT --to-ports 30345
iptables -t nat -A OUTPUT -p tcp --dport 443 -j REDIRECT --to-ports 30345
iptables -t nat -A OUTPUT -p udp --dport 53 -j REDIRECT --to-ports 30053

Thanks again for the help.

Score:1
in flag

If anyone is interested, I found trying to use a node port too difficult, and instead opted to just move transocks to its own pod and use host networking, along with the above forwarding rules to make this work. Here is the manifest I used for the transocks pod:

{
  "apiVersion": "v1",
  "kind": "Pod",
  "metadata": {
    "name": "transocks"
  },
  "spec": {
    "hostNetwork": true,
    "dnsPolicy": "ClusterFirstWithHostNet",
    "containers": [
      {
        "name": "transocks",
        "image": "jusschwa/transocks-proxy"
      }
    ]
  }
}

This is the transocks toml I am using:

listen = "0.0.0.0:12345"

# Connect to HTTP Proxy
proxy_url = "http://127.0.0.1:30128"

[log]
level = "error"

30128 is the nodeport for the squid proxy.

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.