Score:1

OpenVPN connection resetting every 2 minutes

vn flag

I have an OpenVPN server running on Ubuntu in AWS, and using Tunnelblick on macOS to connect to it. I have no problem connecting to other VPN servers, but this one seems to time out/reset every 2 minutes.

My OVPN profile:

client
dev tun
proto udp
remote ............... 1194
resolv-retry infinite
nobind
user nobody
group nogroup
persist-key
persist-tun
remote-cert-tls server
cipher AES-256-CBC
verb 3
cipher AES-128-CBC
auth SHA256
key-direction 1
<ca>
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
</ca>
<cert>
Certificate:
    Data:
        Version: 3 (0x2)
...
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
</cert>
<key>
-----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----
</key>
<tls-auth>
#
# 2048 bit OpenVPN static key
#
-----BEGIN OpenVPN Static key V1-----
...
-----END OpenVPN Static key V1-----
</tls-auth>

On connection, the server pushes the following settings:

PUSH: Received control message: 'PUSH_REPLY,redirect-gateway def1 bypass-dhcp,dhcp-option DNS 8.8.8.8,route 172.16.0.0 255.255.240.0,route 172.16.16.0 255.255.240.0,route 172.16.128.0 255.255.240.0,route 172.16.144.0 255.255.240.0,route 10.8.0.1,topology net30,ping 10,ping-restart 120,ifconfig 10.8.0.6 10.8.0.5,peer-id 1,cipher AES-256-GCM'

(note specifically ping 10,ping-restart 120)

Upping the log level on the client, it does look like the connection is sending data packets:

2021-09-03 11:31:21.848620 UDP WRITE [62] to [AF_INET]...:1194: P_ACK_V1 kid=0 pid=[ #13 ] [ 6 ]
2021-09-03 11:31:21.848768 UDP WRITE [130] to [AF_INET]...:1194: P_DATA_V2 kid=0 DATA len=129
2021-09-03 11:31:21.848856 UDP WRITE [226] to [AF_INET]...:1194: P_DATA_V2 kid=0 DATA len=225

However the connection always dies after about 2 minutes. Client log states:

2021-09-03 11:40:26.121900 [cc-vpn] Inactivity timeout (--ping-restart), restarting
2021-09-03 11:40:26.122379 SIGUSR1[soft,ping-restart] received, process restarting
2021-09-03 11:40:26.122504 MANAGEMENT: >STATE:1630683626,RECONNECTING,ping-restart,,,,,
2021-09-03 11:40:26.448969 MANAGEMENT: CMD 'hold release'

Nothing really in the server log other than a connection restarting.

The 2 minute timeout makes sense given the ping-restart 120 setting pushed to the client, but I'm not clear why it thinks it has been inactive. What am I missing? Is there a setting on the client that stops the ping from being sent to the server properly?

Specifically adding ping/ping-restart to the client config doesn't seem to help (I assume it would be overridden by the server PUSH anyway).

How can I debug this and figure out why the connection isn't being kept alive?

Nikita Kipriyanov avatar
za flag
Could it be you have two clients using the same certificate/key pair and running simultaneously?
DrTeeth avatar
vn flag
Yep. That's it. I had run across a few other answers suggesting that, but I couldn't find the other client, and I knew I didn't have one running. Turns out someone else had an old container running in the background that was still using it. Better yet, both clients tried to reconnect every time they got disconnected, so they were just battling for the connection. I do wish the error message was better... it's not really "inactivity timeout"... @NikitaKipriyanov If you want to write this as an answer, I'll be happy to give you the credit. Otherwise I'll just write up my own stupidity.
Score:1
za flag

This is often the sign that there are more that one client who are using this key/certificate pair:

  • (1) authneticates
  • (2) authenticates; server sees the same certificate, so it thinks it was just replaced connection, and (1) will not receive keepalive pings anymore
  • (1) misses some pings, decides the connection died and reconnects, now (2) won't receive pings
  • (2) misses some pings, decides the connection died and reconnects, now (1) won't receive pings

You see what's going and also it is clear how inactivity timeout set by ping-restart is involved here.

For this to not happen, you have to carefully manage your VPN CA. In particular:

  • Keep track where your keys are installed and who is in charge of the device where each key is installed. Have a way to contact anyone who has active VPN keys (e.g. record their phone number, email, etc., you may set up OpenSSL so it'll ask for that data during certificate issuance and record that data directly into certificates and CA index).
  • Never use the same key/cert more than once; never put key/cert into templates; if you clone some system, clear keys there. Keys must be always generated and certificated issued anew each time when the system is deployed.
  • If some user asks for (another) key/cert while they have an active one, they must explain why. They may have lost old data because OS was reinstalled and they forgot to save VPN configuration; or they simply may need to have VPN on additional computer. Or whatever. Evaluating their explanation, you either first revoke old key before issuing another one, or issue a key with another CN to avoid a clash.
  • Educate your users to always notify you their key/cert is not used anymore (it's lost or the reason for its issuance is lost) so you can revoke it. And you then have to revoke it.
  • Very important, educate users to ugently notify you if they suspect key/cert was stolen, in which case you must immediately revoke it.

These are parts of a process called "network security". VPN couldn't be secure without certain discipline, no matter how perfect its software and state-of-art cryptography it is using.

DrTeeth avatar
vn flag
I really wish either client or server had a better error message - it isn't really an "inactivity timeout", but rather two clients both trying to steal the active connection from each other. That said, this was the answer for me - someone else was using the profile on a machine that neither of us really used anymore, and it was left trying to connect/reconnect while I was trying to do the same.
Nikita Kipriyanov avatar
za flag
Server couldn't to that. It just reports what it sees: a client repeatedly connects again and again. It can't easily associate subsequent connection attempts and decide "these are two clashing clients with same certificate". Each new authentication arrives from new (address,port) pair, so while there could be an *insight* these *might* be just two clients, there is no reliable way to determine that. I've seen cases where this insight was wrong. Nobody wants the server error messages to be *misleading*, so better they stay as they are.
DrTeeth avatar
vn flag
I'm still not convinced the server couldn't do something. Client A connects. Client B then connects, making Client A's connection no longer valid because it uses the same certificate. Only the server knows that it invalidated one connection because another client connected. At that point the server could a) drop a message to it's own log file (I didn't see any), b) send a message to one or both clients that multiple clients are attempting to use the same cert (similar to static DNS conflict messages using the same IP), c) keep track of IPs and send a message to a client that reconnects
DrTeeth avatar
vn flag
But I get that this is a design decision on OpenVPN's side, so outside the scope of this. Just... maybe if an OpenVPN dev happens to be bored one night reading these comments...?
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.