Join Log in
Score:0
avatar Server

Stunnel outgoing packets strangely modified

es flag
r0b0tron

I have a box with two nics setup as bridge. Ebtables redirects http traffic to iptables. The br0 ip address is 10.10.10.10. Stunnel is setup with transparent = source. It accepts connections on 127.1.1.1:8080 and always connects to the same ip address (10.10.20.20) on port 80.

I have the following iptables rules in place:

iptables -t nat -I PREROUTING -p tcp --dport 80 -i ens192 -j DNAT --to-destination 127.1.1.1:8080
iptables -t mangle -N DIVERT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 1
iptables -t mangle -A DIVERT -j ACCEPT

Routing is set up too. If a client connects to the box itself on port 80 everything works. Stunnel connects to the destination (10.10.20.20). But if the client has a different destination address, stunnel still trys to connect to 10.10.20.20, but can't.

So when i trace the raw Packets to 10.10.20.20, i can see different behaviours. The expected one:

trace id 71a8325b ip raw OUTPUT packet: oif "br0" ip saddr 10.10.10.10 ip daddr 10.10.20.20 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 28971 ip length 60 tcp sport 51408 tcp dport 80 tcp flags == syn tcp window 64240
trace id 71a8325b ip raw OUTPUT rule meta l4proto tcp ip daddr 10.10.20.20 counter packets 37 bytes 3265 meta nftrace set 1 (verdict continue)
trace id 71a8325b ip raw OUTPUT verdict continue
trace id 71a8325b ip raw OUTPUT policy accept
trace id 71a8325b ip filter OUTPUT packet: oif "br0" ip saddr 10.10.10.10 ip daddr 10.10.20.20 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 28971 ip length 60 tcp sport 51408 tcp dport 80 tcp flags == syn tcp window 64240
trace id 71a8325b ip filter OUTPUT verdict continue
trace id 71a8325b ip filter OUTPUT policy accept
trace id 71a8325b inet filter output packet: oif "br0" ip saddr 10.10.10.10 ip daddr 10.10.20.20 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 28971 ip protocol tcp ip length 60 tcp sport 51408 tcp dport 80 tcp flags == syn tcp window 64240
trace id 71a8325b inet filter output verdict continue
trace id 71a8325b inet filter output policy accept

And the unexpected one, where stunnel cannot connect:

trace id fd9543bc ip raw OUTPUT packet: oif "br0" ip saddr 10.10.10.10 ip daddr 10.10.20.20 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 26448 ip length 60 tcp sport 34188 tcp dport 80 tcp flags == syn tcp window 64240
trace id fd9543bc ip raw OUTPUT rule meta l4proto tcp ip daddr 10.10.20.20 counter packets 52 bytes 4540 meta nftrace set 1 (verdict continue)
trace id fd9543bc ip raw OUTPUT verdict continue
trace id fd9543bc ip raw OUTPUT policy accept
trace id fd9543bc ip filter OUTPUT packet: oif "br0" ip saddr 10.10.10.10 ip daddr 127.1.1.1 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 26448 ip length 60 tcp sport 34188 tcp dport 8080 tcp flags == syn tcp window 64240
trace id fd9543bc ip filter OUTPUT verdict continue
trace id fd9543bc ip filter OUTPUT policy accept
trace id fd9543bc inet filter output packet: oif "br0" ip saddr 10.10.10.10 ip daddr 127.1.1.1 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 26448 ip protocol tcp ip length 60 tcp sport 34188 tcp dport 8080 tcp flags == syn tcp window 64240
trace id fd9543bc inet filter output verdict continue
trace id fd9543bc inet filter output policy accept

It seems like the destination address gets destination natted. But i cannot get why and when. I only DNAT in the nat table PREROUTING. As of my understanding, this packet should not hit this rule again in any way. And why does this only happen, when the original destination wasn't the boxes own ip-address? I am thinking, that maybe stunnel changes the packet itself?

Here is the full iptables-save output

# Generated by iptables-save v1.8.7 on Thu Nov 18 22:40:01 2021
*nat
:PREROUTING ACCEPT [14:1295]
:INPUT ACCEPT [14:1295]
:OUTPUT ACCEPT [2:196]
:POSTROUTING ACCEPT [4:316]
-A PREROUTING -i ens192 -p tcp -m tcp --dport 80 -j DNAT --to-destination 127.1.1.1:8080
COMMIT
# Completed on Thu Nov 18 22:40:01 2021
# Generated by iptables-save v1.8.7 on Thu Nov 18 22:40:01 2021
*mangle
:PREROUTING ACCEPT [15:1154]
:INPUT ACCEPT [172:24172]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [222:44999]
:POSTROUTING ACCEPT [222:44999]
:DIVERT - [0:0]
-A PREROUTING -p tcp -m socket -j DIVERT
-A DIVERT -j MARK --set-xmark 0x1/0xffffffff
-A DIVERT -j ACCEPT
COMMIT
# Completed on Thu Nov 18 22:40:01 2021

Any ideas, how i could go on with this?

60
0 + 0
A.B avatar
cl flag
A.B
Looks like this other question becomes obsolete? https://serverfault.com/questions/1083810/iptables-modify-output-flow
0
Reply
es flag
r0b0tron
No, the other question is focused on possibilities to workaround with iptables.
0
Reply
Nikita Kipriyanov avatar
za flag
Nikita Kipriyanov
According to the [packet flow](https://upload.wikimedia.org/wikipedia/commons/3/37/Netfilter-packet-flow.svg), any externally produced packet which is sent into routing from `broute`, should hit the `nat PREROUTING` rule (once, because nat table is traversed only for new connections), but locally generated packet has no way to hit that rule, so the behaviour you see is expected.
0
Reply
es flag
r0b0tron
No it's not, since the packets that seem to get DNATed are the ones that are generated locally and therefore only going through the OUTPUT chain.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.