Score:1

bypass vpn for a specific port (possible iptables malfunction)

mn flag

I have a linux box which is an openVPN client all its traffic goes through the tunel , i have an nginx server that i want to go through eth0 and what i tried to do is to reroute all traffic on port 80 through eth0 via 192.168.1.1

root@digger02:~# ip route
0.0.0.0/1 via 10.8.0.1 dev tun0
default via 192.168.1.1 dev eth0 proto dhcp src 192.168.1.154 metric 10
10.8.0.0/24 dev tun0 proto kernel scope link src 10.8.0.2
XXX.XXX.XXX.XXX via 192.168.1.1 dev eth0  
128.0.0.0/1 via 10.8.0.1 dev tun0
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.154
192.168.1.1 dev eth0 proto dhcp scope link src 192.168.1.154 metric 10

where XXX.XXX.XXX.XXX is my VPN server`s public IP address

root@digger02:~# iptables -t mangle -L -v -n
Chain PREROUTING (policy ACCEPT 665K packets, 164M bytes)
 pkts bytes target     prot opt in     out     source               destination
  176  8430 MARK       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80 MARK set 0x80

Chain INPUT (policy ACCEPT 665K packets, 164M bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 715K packets, 91M bytes)
 pkts bytes target     prot opt in     out     source               destination
   35 82961 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            cgroup 1114129 MARK set 0xb

Chain POSTROUTING (policy ACCEPT 715K packets, 91M bytes)
 pkts bytes target     prot opt in     out     source               destination
root@digger02:~# cat /etc/iproute2/rt_tables
#
# reserved values
#
255     local
254     main
253     default
0       unspec
#
# local
#
#1      inr.ruhep
80 http
11 novpn

this is what i tried to do

iptables -t mangle -A PREROUTING -p tcp --dport 80 -j MARK --set-mark 0x80
echo "80 http" >> /etc/iproute2/rt_tables
ip route add default via 192.168.1.1 dev eth0 table http
ip rule add fwmark 0x80 lookup 80
ip route flush table 100
ip route flush cache
ip rule add from xxx.xxx.xxx.xxx table 100
ip route add table 100 to 192.168.1.0/24 dev eth0
ip route add table 100 default via 192.168.1.1

where table 100 is created by OpenVPN's route-up script and xxx.xxx.xxx.xxx is my public ip

nothing works for me , i even tried this script https://gist.github.com/kriswebdev/a8d291936fe4299fb17d3744497b1170 which basically marks all traffic for an application

I feel that iptables may not be working on my system at all, how do I diagnose that

A.B avatar
cl flag
A.B
It's not very clear where the different system are and are used vs each others: where is the nginx server? in the same LAN? on the remote side of the VPN? etc. It would help if you added for example an ascii art schematic.
raga70 avatar
mn flag
the nginx server is on the same machine (openVPN client) and i want the traffic on port 80 to bypass the tunnel connection
Score:0
cl flag
A.B

Since Linux kernel 4.17, there's no need to use iptables at all: the routing stack can directly choose a different route depending on port without having it marked, rerouted and requiring a NAT band-aid (but usually only for UDP or for TCP as client, not for TCP as server like here):

With the nginx server running on the VPN client, assuming there's no iptables ruleset and no additional routing rule yet in place...

Add the alternate route in the routing table, preceded by the LAN route before (it will at least prevent connections from LAN to uselessly go through the router which will have to emit ICMP redirects). This table will be flushed if the interface goes down then up, so this should be integrated with the tool managing the interface:

ip route add 192.168.1.0/24 dev eth0 table 80
ip route add default via 192.168.1.1 dev eth0 table 80

Choose locally initiated (that's the special role of iif lo here) reply traffic from port 80 (to anywhere) to use this table:

ip rule add iif lo ipproto tcp sport 80 lookup 80

Same for incoming HTTP traffic, ie the original direction (not really needed since SRPF won't be enabled, but for consistency anyway):

ip rule add iif eth0 ipproto tcp dport 80 lookup 80

Loosen SRPF in case it was enabled because I'm not certain TCP related traffic such as ICMP errors during PMTUD might be accepted in case it was set as strict:

sysctl -w net.ipv4.conf.eth0.rp_filter=2

HTTP/3 support (which uses QUIC and is thus over UDP) just requires the same rules for UDP:

ip rule add iif lo ipproto udp sport 80 lookup 80
ip rule add iif eth0 ipproto udp dport 80 lookup 80

Should some IP addresses be made an exception to this exception on port 80, one method among others is to add them in the routing table as not matching using the throw keyword, thus falling through to the main routing table. For example to have traffic from 10.8.0.0/24 which is guaranteed to be OpenVPN traffic, to still work with nginx:

throw - a special control route used together with policy rules. If such a route is selected, lookup in this table is terminated pretending that no route was found. Without policy routing it is equivalent to the absence of the route in the routing table. The packets are dropped and the ICMP message net unreachable is generated. The local senders get an ENETUNREACH error.

ip route add throw 10.8.0.0/24 table 80
raga70 avatar
mn flag
thank you vary much
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.