Join Log in
Score:0
Robert Driller avatar Server

OpenVPN routing between subnets and LAN not working

cn flag
Robert Driller

I have set up an openVPN server with topology subnet. It contains a set of subnets for clients in the region 10.0.X.X which are routed. In the Server network there is a client (not in VPN) that runs a service that the VPN users need to access.

So basically I try to give different user groups access to a webservice. Both openvpn and the webservice run in docker instances

ANY HELP OR TIP IS VERY MUCH APPRECIATED SINCE I AM FIGHTING WITH THIS SINCE 2 WEEKS NOW

Steps Taken: I setup openvpn conf with routes and with client-to-client

server 192.168.255.0 255.255.255.0
verb 3
key /etc/openvpn/pki/private/xxxx.key
ca /etc/openvpn/pki/ca.crt
cert /etc/openvpn/pki/issued/xxxx.crt
dh /etc/openvpn/pki/dh.pem
tls-auth /etc/openvpn/pki/ta.key
key-direction 0
keepalive 10 60
persist-key
persist-tun

proto udp
# Rely on Docker to do port mapping, internally always 1194
port 1194
dev tun
status /tmp/openvpn-status.log
topology subnet
client-config-dir ccd

user nobody
group nogroup
comp-lzo no
client-to-client

### Route Configurations Below
route 192.168.254.0 255.255.255.0
route 10.0.0.0 255.255.255.0
route 10.0.1.0 255.255.255.0
route 10.0.3.0 255.255.255.0
route 10.0.4.0 255.255.255.0
route 10.0.5.0 255.255.255.0

### Push Configurations Below
#push "block-outside-dns"
push "dhcp-option DNS 8.8.8.8"
push "dhcp-option DNS 8.8.4.4"
push "comp-lzo no"
push "route 172.17.0.0 255.255.255.0"

I configured forwarding by setting net.ipv4.ip_forward = 1 in /etc/sysctl.conf

I configured the iptable rules

-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-A FORWARD -s 192.168.255.0/24 -d 172.17.0.0/24 -i tun0 -j ACCEPT
-A FORWARD -j ACCEPT
-A FORWARD -s 192.168.255.0/24 -d 10.0.0.0/24 -i tun0 -j ACCEPT

But from the IP in the server range (192.168.255.2) am unable to ping or access anything (172.17.0.4 or 10.0.0.1)

A traceroute ends at the gateway

% traceroute 10.0.0.1
traceroute to 10.0.0.1 (10.0.0.1), 64 hops max, 52 byte packets
1 192.168.88.1 (192.168.88.1) 12.279 ms 3.251 ms 1.882 ms
2 *

here also the log from my client

2022-04-30 06:24:38.983758 *Tunnelblick: macOS 12.3.1 (21E258); Tunnelblick 3.8.5beta05 (build 5650)
2022-04-30 06:24:39.446714 *Tunnelblick: Attempting connection with greenhive_master using shadow copy; Set nameserver = 769; monitoring connection
2022-04-30 06:24:39.450786 *Tunnelblick: openvpnstart start greenhive_master.tblk 52399 769 0 1 0 34652464 -ptADGNWradsgnw 2.4.10-openssl-1.1.1j
2022-04-30 06:24:39.477788 *Tunnelblick: openvpnstart starting OpenVPN
2022-04-30 06:24:39.857743 OpenVPN 2.4.10 x86_64-apple-darwin [SSL (OpenSSL)] [LZO] [LZ4] [PKCS11] [MH/RECVDA] [AEAD] built on Feb 25 2021
2022-04-30 06:24:39.857947 library versions: OpenSSL 1.1.1j  16 Feb 2021, LZO 2.10
2022-04-30 06:24:39.859534 MANAGEMENT: TCP Socket listening on [AF_INET]127.0.0.1:52399
2022-04-30 06:24:39.859562 Need hold release from management interface, waiting...
2022-04-30 06:24:40.078894 *Tunnelblick: openvpnstart log:
     OpenVPN started successfully.
     Command used to start OpenVPN (one argument per displayed line):
          /Applications/Tunnelblick.app/Contents/Resources/openvpn/openvpn-2.4.10-openssl-1.1.1j/openvpn
          --daemon
          --log /Library/Application Support/Tunnelblick/Logs/-SUsers-Srobertk-SLibrary-SApplication Support-STunnelblick-SConfigurations-Sgreenhive_master.tblk-SContents-SResources-Sconfig.ovpn.769_0_1_0_34652464.52399.openvpn.log
          --cd /Library/Application Support/Tunnelblick/Users/robertk/greenhive_master.tblk/Contents/Resources
          --machine-readable-output
          --setenv IV_GUI_VER "net.tunnelblick.tunnelblick 5650 3.8.5beta05 (build 5650)"
          --verb 3
          --config /Library/Application Support/Tunnelblick/Users/robertk/greenhive_master.tblk/Contents/Resources/config.ovpn
          --setenv TUNNELBLICK_CONFIG_FOLDER /Library/Application Support/Tunnelblick/Users/robertk/greenhive_master.tblk/Contents/Resources
          --verb 3
          --cd /Library/Application Support/Tunnelblick/Users/robertk/greenhive_master.tblk/Contents/Resources
          --management 127.0.0.1 52399 /Library/Application Support/Tunnelblick/geeielmngfddkiiidnhcaaaogadlpdifnpjaepip.mip
          --management-query-passwords
          --management-hold
          --script-security 2
          --route-up /Applications/Tunnelblick.app/Contents/Resources/client.up.tunnelblick.sh -9 -d -f -m -w -ptADGNWradsgnw
          --down /Applications/Tunnelblick.app/Contents/Resources/client.down.tunnelblick.sh -9 -d -f -m -w -ptADGNWradsgnw
2022-04-30 06:24:40.108790 MANAGEMENT: Client connected from [AF_INET]127.0.0.1:52399
2022-04-30 06:24:40.136505 MANAGEMENT: CMD 'pid'
2022-04-30 06:24:40.136565 MANAGEMENT: CMD 'auth-retry interact'
2022-04-30 06:24:40.136595 MANAGEMENT: CMD 'state on'
2022-04-30 06:24:40.136615 MANAGEMENT: CMD 'state'
2022-04-30 06:24:40.136655 MANAGEMENT: CMD 'bytecount 1'
2022-04-30 06:24:40.138068 *Tunnelblick: Established communication with OpenVPN
2022-04-30 06:24:40.152091 *Tunnelblick: >INFO:OpenVPN Management Interface Version 1 -- type 'help' for more info
2022-04-30 06:24:40.154175 MANAGEMENT: CMD 'hold release'
2022-04-30 06:24:40.155529 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
2022-04-30 06:24:40.161486 Outgoing Control Channel Authentication: Using 160 bit message hash 'SHA1' for HMAC authentication
2022-04-30 06:24:40.161545 Incoming Control Channel Authentication: Using 160 bit message hash 'SHA1' for HMAC authentication
2022-04-30 06:24:40.161718 MANAGEMENT: >STATE:1651292680,RESOLVE,,,,,,
2022-04-30 06:24:40.259123 TCP/UDP: Preserving recently used remote address: [AF_INET]xx.xx.xx.xx:1194
2022-04-30 06:24:40.259272 Socket Buffers: R=[786896->786896] S=[9216->9216]
2022-04-30 06:24:40.259296 UDP link local: (not bound)
2022-04-30 06:24:40.259311 UDP link remote: [AF_INET]xx.xx.xx.xx:1194
2022-04-30 06:24:40.259354 MANAGEMENT: >STATE:1651292680,WAIT,,,,,,
2022-04-30 06:24:40.305386 MANAGEMENT: >STATE:1651292680,AUTH,,,,,,
2022-04-30 06:24:40.305631 TLS: Initial packet from [AF_INET]xx.xx.xx.xx:1194, sid=06c4262e 884c5cdc
2022-04-30 06:24:40.369493 VERIFY OK: depth=1, CN=greenhive
2022-04-30 06:24:40.370236 VERIFY KU OK
2022-04-30 06:24:40.370275 Validating certificate extended key usage
2022-04-30 06:24:40.370301 ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication
2022-04-30 06:24:40.370323 VERIFY EKU OK
2022-04-30 06:24:40.370346 VERIFY OK: depth=0, CN=VPN.greenhive.at
2022-04-30 06:24:40.439317 WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1541', remote='link-mtu 1542'
2022-04-30 06:24:40.439596 WARNING: 'comp-lzo' is present in remote config but missing in local config, remote='comp-lzo'
2022-04-30 06:24:40.439767 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, 2048 bit RSA
2022-04-30 06:24:40.439834 [VPN.greenhive.at] Peer Connection Initiated with [AF_INET]xx.xx.xx.xx:1194
2022-04-30 06:24:41.558800 MANAGEMENT: >STATE:1651292681,GET_CONFIG,,,,,,
2022-04-30 06:24:41.559492 SENT CONTROL [VPN.greenhive.at]: 'PUSH_REQUEST' (status=1)
2022-04-30 06:24:41.652600 PUSH: Received control message: 'PUSH_REPLY,dhcp-option DNS 8.8.8.8,dhcp-option DNS 8.8.4.4,comp-lzo no,route 172.17.0.0 255.255.255.0,route-gateway 192.168.255.1,topology subnet,ping 10,ping-restart 60,ifconfig 192.168.255.2 255.255.255.0,peer-id 1,cipher AES-256-GCM'
2022-04-30 06:24:41.652921 OPTIONS IMPORT: timers and/or timeouts modified
2022-04-30 06:24:41.652958 OPTIONS IMPORT: compression parms modified
2022-04-30 06:24:41.652985 OPTIONS IMPORT: --ifconfig/up options modified
2022-04-30 06:24:41.653008 OPTIONS IMPORT: route options modified
2022-04-30 06:24:41.653030 OPTIONS IMPORT: route-related options modified
2022-04-30 06:24:41.653051 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
2022-04-30 06:24:41.653072 OPTIONS IMPORT: peer-id set
2022-04-30 06:24:41.653094 OPTIONS IMPORT: adjusting link_mtu to 1624
2022-04-30 06:24:41.653115 OPTIONS IMPORT: data channel crypto options modified
2022-04-30 06:24:41.653139 Data Channel: using negotiated cipher 'AES-256-GCM'
2022-04-30 06:24:41.653816 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
2022-04-30 06:24:41.653851 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
2022-04-30 06:24:41.654722 Opening utun (connect(AF_SYS_CONTROL)): Resource busy (errno=16)
2022-04-30 06:24:41.654977 Opening utun (connect(AF_SYS_CONTROL)): Resource busy (errno=16)
2022-04-30 06:24:41.655036 Opening utun (connect(AF_SYS_CONTROL)): Resource busy (errno=16)
2022-04-30 06:24:41.655181 Opened utun device utun3
2022-04-30 06:24:41.655203 MANAGEMENT: >STATE:1651292681,ASSIGN_IP,,192.168.255.2,,,,
2022-04-30 06:24:41.655217 /sbin/ifconfig utun3 delete
                           ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address
2022-04-30 06:24:41.665109 NOTE: Tried to delete pre-existing tun/tap instance -- No Problem if failure
2022-04-30 06:24:41.666818 /sbin/ifconfig utun3 192.168.255.2 192.168.255.2 netmask 255.255.255.0 mtu 1500 up
2022-04-30 06:24:41.672645 /sbin/route add -net 192.168.255.0 192.168.255.2 255.255.255.0
                           add net 192.168.255.0: gateway 192.168.255.2
2022-04-30 06:24:41.679237 MANAGEMENT: >STATE:1651292681,ADD_ROUTES,,,,,,
2022-04-30 06:24:41.679318 /sbin/route add -net 172.17.0.0 192.168.255.1 255.255.255.0
                           add net 172.17.0.0: gateway 192.168.255.1
                           06:24:41 *Tunnelblick:  **********************************************
                           06:24:41 *Tunnelblick:  Start of output from client.up.tunnelblick.sh
                           06:24:43 *Tunnelblick:  Retrieved from OpenVPN: name server(s) [ 8.8.8.8 8.8.4.4 ], search domain(s) [ ] and SMB server(s) [ ] and using default domain name [ openvpn ]
                           06:24:44 *Tunnelblick:  WARNING: Ignoring DomainName 'openvpn' because DomainName was set manually and '-allowChangesToManuallySetNetworkSettings' was not specified
                           06:24:44 *Tunnelblick:  WARNING: Ignoring ServerAddresses '8.8.8.8 8.8.4.4' because ServerAddresses was set manually and '-allowChangesToManuallySetNetworkSettings' was not specified
                           06:24:44 *Tunnelblick:  Setting search domains to '8.8.8.8 8.8.4.4' because the search domains were not set manually (or are allowed to be changed) and 'Prepend domain name to search domains' was not selected
                           06:24:45 *Tunnelblick:  Saved the DNS and SMB configurations so they can be restored
                           06:24:45 *Tunnelblick:  Did not change DNS ServerAddresses setting of '8.8.8.8 8.8.4.4' (but re-set it)
                           06:24:45 *Tunnelblick:  Changed DNS SearchDomains setting from 'openvpn' to '8.8.8.8 8.8.4.4'
                           06:24:45 *Tunnelblick:  Changed DNS DomainName setting from '' to '8.8.8.8 8.8.4.4'
                           06:24:45 *Tunnelblick:  Did not change SMB NetBIOSName setting of ''
                           06:24:45 *Tunnelblick:  Did not change SMB Workgroup setting of ''
                           06:24:45 *Tunnelblick:  Did not change SMB WINSAddresses setting of ''
                           06:24:45 *Tunnelblick:  DNS servers '8.8.8.8 8.8.4.4' were set manually
                           06:24:45 *Tunnelblick:  DNS servers '8.8.8.8 8.8.4.4' will be used for DNS queries when the VPN is active
                           06:24:45 *Tunnelblick:  The DNS servers include only free public DNS servers known to Tunnelblick.
                           06:24:45 *Tunnelblick:  Flushed the DNS cache via dscacheutil
                           06:24:45 *Tunnelblick:  /usr/sbin/discoveryutil not present. Not flushing the DNS cache via discoveryutil
                           06:24:45 *Tunnelblick:  Notified mDNSResponder that the DNS cache was flushed
                           06:24:45 *Tunnelblick:  Not notifying mDNSResponderHelper that the DNS cache was flushed because it is not running
                           06:24:45 *Tunnelblick:  Setting up to monitor system configuration with process-network-changes
                           06:24:45 *Tunnelblick:  End of output from client.up.tunnelblick.sh
                           06:24:45 *Tunnelblick:  **********************************************
2022-04-30 06:24:45.352811 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
2022-04-30 06:24:45.352830 Initialization Sequence Completed
2022-04-30 06:24:45.352845 MANAGEMENT: >STATE:1651292685,CONNECTED,SUCCESS,192.168.255.2,xx.xx.xx.xx,1194,,
2022-04-30 06:24:46.571157 *Tunnelblick: Routing info stdout:
   route to: 8.8.4.4
destination: 8.8.4.4
    gateway: 192.168.88.1
  interface: en0
      flags: <UP,GATEWAY,HOST,DONE,WASCLONED,IFSCOPE,IFREF,GLOBAL>
 recvpipe  sendpipe  ssthresh  rtt,msec    rttvar  hopcount      mtu     expire
       0         0         0         0         0         0      1500         0 
stderr:

2022-04-30 06:24:46.593014 *Tunnelblick: Warning: DNS server Address 8.8.4.4 is a known public DNS server but is not being routed through the VPN
2022-04-30 06:24:46.680197 *Tunnelblick: Routing info stdout:
   route to: 8.8.8.8
destination: 8.8.8.8
    gateway: 192.168.88.1
  interface: en0
      flags: <UP,GATEWAY,HOST,DONE,WASCLONED,IFSCOPE,IFREF,GLOBAL>
 recvpipe  sendpipe  ssthresh  rtt,msec    rttvar  hopcount      mtu     expire
       0         0         0         0         0         0      1500         0 
stderr:

2022-04-30 06:24:46.705581 *Tunnelblick: Warning: DNS server Address 8.8.8.8 is a known public DNS server but is not being routed through the VPN

EDIT: here the ip route of the server

bash-5.0# ip route 
default via 172.17.0.1 dev eth0 
10.0.0.0/24 via 192.168.255.2 dev tun0 
10.0.1.0/24 via 192.168.255.2 dev tun0 
10.0.3.0/24 via 192.168.255.2 dev tun0 
172.17.0.0/16 dev eth0 proto kernel scope link src 
172.17.0.2 192.168.254.0/24 via 192.168.255.2 dev tun0 192.168.255.0/24 dev tun0 proto kernel scope link src 192.168.255.1

Of the client

172.17/24          192.168.255.2      UGSc            utun3

Statuslog

bash-5.0# cat /tmp/openvpn-status.log
OpenVPN CLIENT LIST
Updated,Sun May  1 08:20:48 2022
Common Name,Real Address,Bytes Received,Bytes Sent,Connected Since
at_dev_1,179.115.236.15:34846,23335,23021,Sun May  1 07:14:26 2022
master,179.115.236.15:64773,9574,9889,Sun May  1 07:55:44 2022
ROUTING TABLE
Virtual Address,Common Name,Real Address,Last Ref
192.168.255.2,master,179.115.236.15:64773,Sun May  1 07:55:44 2022
10.0.0.1,at_dev_1,179.115.236.15:34846,Sun May  1 07:14:26 2022
GLOBAL STATS
Max bcast/mcast queue length,1
END

EDIT here an image to show the situation Graphical explanation

80
0 + 0
Nikita Kipriyanov avatar
za flag
Nikita Kipriyanov
Please show `ip route` output of affected systems when VPN connection is established, and output of `status` command in the OpenVPN management interface (to show internal routes and other state). You can mask public IPs, but absolutely keep all private addresses as they are. I also want to note that `topology` has nothing to do with the way you lay out networks outside OpenVPN. It only sets up its internal workings. The `topology subnet` mode makes OpenVPN itself a router and that requires setting up *internal* routes in it, with the help of `iroute` commands that go into CCD files.
0
Reply
Nikita Kipriyanov avatar
za flag
Nikita Kipriyanov
Please, put your routing output into the question by [editing](https://serverfault.com/posts/1099885/edit) and remove from the comment; it breaks newlines in the comment so it's unreadable. Also I asked a *management* `status` output. It shows internals which we need here. It's crucial in `subnet` mode, because it'll show a routing table internal to the OpenVPN process itself. // Notice, I've talked about `iroute`, not about push route and so on. Read `man openvpn` regarding that command and what it does.
0
Reply
Robert Driller avatar
cn flag
Robert Driller
Sorry about that, I adjusted it. I also did some more reading on iroute. Blease correct me if i am wrong but i understand it to be a special routing in case i want to connect to an address that is "accessible" cia a VPN **client**. But since the address I want to contact is in the LAN of the VPN **server** and i want all clients to be able to contact it, isn't push route the right way to go? I got this from https://backreference.org/2009/11/15/openvpn-and-iroute/index.html
0
Reply
Nikita Kipriyanov avatar
za flag
Nikita Kipriyanov
Yes, I am writing an answer with explanations. It's not quite "hidden behind the client", but it's routing that happens inside the OpenVPN process on the server.
0
Reply
Score:0
Nikita Kipriyanov avatar Server
za flag
Nikita Kipriyanov

So we see you don't have any internal routes inside OpenVPN. That probably comes from the fact you don't use any iroute statements in your configuration. That's why only the VPN addresses of the server and clients are accessible to each other.

The general structure of the OpenVPN "topology subnet" VPN looks like this from the networking (OSI L3) perspective:

                                          192.168.255.10[tun] (client2) [eth] 10.0.2.1 --- ...
                                                   |
                                              [client2].9                         [eth] 10.0.1.1 --- ...
(server) [tun]192.168.255.1 --- .2[server] (OpenVPN process) [client1].5 --- 192.168.255.6[tun] (client1)
[eth] 10.0.0.1 --- ...                       .13[client3]
                                                   |
         ... --- 10.0.3.1 [eth] (client3) [tun]192.168.255.14

In this example with three clients, OpenVPN process looks like a router with four interfaces (one is facing towards the server). This router does need routing setup too! This is what iroute keyword does.

So, for clients to reach a network 10.0.0.0/24 behind the server, they need a route to that network through their respective OpenVPN client-facing "address" (for example, client1 needs ip route add 10.0.0.0/24 via 192.168.255.5). These routes you push with "push route". To reach networks behind other clients, you make similar routes; in my example case, these network are consecutive and all of them could be condensed into the single push "route 10.0.0.0 255.255.252.0".

Also the OpenVPN "router" needs appropriate routes to these networks via "client" addresses. For example, 10.0.1.0/24 network should be routed via 192.168.255.6. To do so, a server needs to run an iroute 10.0.1.0 255.255.255.0 on behalf of the client1. To achieve this, you put

iroute 10.0.1.0 255.255.255.0

into client1 file in the client config directory (CCD), so it gets executed right after the client with the common name client1 successfully authenticates.

Networks behind the server don't need iroute. It seems it uses server "interface" as the gateway of last resort.

0 + 0
Robert Driller avatar
cn flag
Robert Driller
Thank you very much for the detailed answer, just to make sure I understood correct. The routing information on the Client side via the push route is not enough, the server needs routing too by using iroute directly in the server config. I added the iroute like this `iroute 172.17.0.0 255.255.255.0` in the server config, but when spinning up openVPN it tells me that : `Options error: option 'iroute' cannot be used in this context (/etc/openvpn/openvpn.conf) Use --help for more information.` Did I missunderstand your last paragraph?
0
Reply
Nikita Kipriyanov avatar
za flag
Nikita Kipriyanov
Yes, because it's wrong. It reminds me that I should always re-read manual, even if I already read it many of times. It seems it routes via server by defaut, e.g. it uses a server as a gateway of last resort. This would explain why server subnets don't need iroutes. What's regarding your original problem is that you gave your clients addresses incompartible with VPN network somehow. Subnets are /30 networks carved from the VPN address space; if you chose `192.168.255.1 255.255.255.0" as the server IP, your clients will have 192.168.255.6/30, .10, .14, etc. assigned to their `tun` interface.
0
Reply
Robert Driller avatar
cn flag
Robert Driller
Sorry you lost me there. I understand that each of my subnets (10.0.1.0/24, 10.0.2.0/24, 10.0.3.0/24 etc.) has a tun interface assigned with /30 space. Would that mean that only 127 subnet's are possible since each reserves 2 addresses? And if all communication is routed via the server, then why would it not route my call to the 172.14.0.4 address behind the oVPN-server?
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.