Score:0

Ubuntu

UFW blocking existing outbound connections when enabled

CpnCrunch

6/24/23, 11:05 PM

I'm having a problem where ufw seems to be blocking existing outbound connections on port 443 when it is enabled. Example:

Feb 24 17:53:00 server5 kernel: [18571501.131985] [UFW BLOCK] IN=eno1 OUT= MAC=d0:50:99:db:0a:be:00:6b:f1:17:4a:81:08:00 SRC=35.196.37.91 DST=1.2.3.4 LEN=40 TOS=0x00 PREC=0x60 TTL=51 ID=24902 DF PROTO=TCP SPT=443 DPT=44496 WINDOW=0 RES=0x00 RST URGP=0 
Feb 24 17:33:40 server5 kernel: [18570340.976130] [UFW BLOCK] IN=eno1 OUT= MAC=d0:50:99:db:0a:be:00:6b:f1:17:4a:81:08:00 SRC=52.10.136.43 DST=1.2.3.4 LEN=83 TOS=0x00 PREC=0x00 TTL=228 ID=23746 DF PROTO=TCP SPT=443 DPT=59404 WINDOW=118 RES=0x00 ACK PSH URGP=0
Feb 27 00:47:07 server5 kernel: [18769144.299731] [UFW BLOCK] IN=eno1 OUT= MAC=d0:50:99:db:0a:be:00:6b:f1:17:4a:81:08:00 SRC=35.196.37.91 DST=1.2.3.4 LEN=1460 TOS=0x00 PREC=0x60 TTL=51 ID=60877 DF PROTO=TCP SPT=443 DPT=42030 WINDOW=229 RES=0x00 ACK URGP=0

Also getting some UDP packets blocked even though I've specifically allowed UDP from 1025-65535:

Feb 24 17:52:19 server5 kernel: [18571459.414576] [UFW BLOCK] IN=eno1 OUT= MAC=d0:50:99:db:0a:be:00:6b:f1:17:4a:81:08:00 SRC=5.6.7.8 DST=1.2.3.4 LEN=69 TOS=0x00 PREC=0x00 TTL=44 ID=59557 PROTO=UDP SPT=58678 DPT=49900 LEN=49

(I've replaced our server ip with 1.2.3.4). The blocked traffic are outgoing curl connections to google drive and Vimeo.

Here is how I set it up:

ufw reset
ufw default allow outgoing
ufw default deny incoming
ufw allow from 96.54.177.7 proto tcp to any port 22
ufw allow from 50.70.255.166 proto tcp to any port 22
ufw allow 443/tcp
ufw allow 80/tcp
ufw allow 25/tcp
ufw allow 587/tcp
ufw allow 1025:65535/udp

ufw status shows:

Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW IN    99.99.99.99               
22/tcp                     ALLOW IN    99.99.99.99             
443/tcp                    ALLOW IN    Anywhere                  
80/tcp                     ALLOW IN    Anywhere                  
25/tcp                     ALLOW IN    Anywhere                  
587/tcp                    ALLOW IN    Anywhere

In testing:

starting a new upload to Vimeo after enabling ufw works fine. Nothing seems to be blocked.
enabling ufw in the middle of a Vimeo upload seems to break it.
telnetting to port 587 (mail) from the server to somewhere else and enabling ufw doesn't seem to cause any problems. The connection remains open and I can type help, etc.
conntrack doesn't ever show outbound connections, but does show inbound connections ok.
when I test on a new ubuntu 20.04 cloud server instance, there are no problems...I see no packets blocked for port 443, and the uploads work fine. But on the test cloud server conntrack is not installed, and even after I install conntrack and conntrackd I don't see any connections at all listed in "conntrack -L".

So, I'm a little confused about what exactly is going on here and whether I should be worried about it. I don't really want to enable ufw until I fully understand what it is going to do to my traffic. How exactly does it keep track of outbound connections if conntrack doesn't track them?

I think there may be a few things going on here, but I'd like to understand why I'm seeing these. The UDP and ACK blocks are the most worrying, but they only seem to happen for a fraction of a second after enabling ufw, so I'm wondering if there is a slight delay while ufw is enabling the rules. The other (RST) may just be due to the connection being closed. The ACK blocks seem to be causing problems with any existing open outbound connections that are actively sending data when the firewall is enabled.

0 + 0

ufw

Score:0

Ubuntu

Doug Smythies

6/25/23, 12:24 AM

For this one:

Feb 24 17:53:00 server5 kernel: [18571501.131985] [UFW BLOCK] IN=eno1 OUT= MAC=d0:50:99:db:0a:be:00:6b:f1:17:4a:81:08:00 SRC=35.196.37.91 DST=1.2.3.4 LEN=40 TOS=0x00 PREC=0x60 TTL=51 ID=24902 DF PROTO=TCP SPT=443 DPT=44496 WINDOW=0 RES=0x00 RST URGP=0

That is not unusual. See more detail in one of my previous answers here.

The next one:

Feb 24 17:33:40 server5 kernel: [18570340.976130] [UFW BLOCK] IN=eno1 OUT= MAC=d0:50:99:db:0a:be:00:6b:f1:17:4a:81:08:00 SRC=52.10.136.43 DST=1.2.3.4 LEN=83 TOS=0x00 PREC=0x00 TTL=228 ID=23746 DF PROTO=TCP SPT=443 DPT=59404 WINDOW=118 RES=0x00 ACK PSH URGP=0

Is not so clear, but also there have been issues with connection dropping upon transition from UFW disabled to enabled in the past. Some investigation methods are provided in another previous answer of mine here

For this one:

conntrack doesn't ever show outbound connections, but does show inbound connections ok.

That is not true. Here is an example:

doug@s15:~$ sudo conntrack -L | grep 192.168.111.134
tcp      6 35972 ESTABLISHED src=192.168.111.1 dst=192.168.111.134 sport=36866 dport=22 src=192.168.111.134 dst=192.168.111.1 sport=22 dport=36866 [ASSURED] mark=0 use=1
conntrack v1.4.6 (conntrack-tools): 68 flow entries have been shown.

That is an outgoing connection made from my main gateway server at 192.168.111.1 to a raspberry pi at 192.168.111.134.

0 + 0

CpnCrunch

6/25/23, 4:57 PM

For my server I don't seem to see outbound connections using conntrack -L. The connections associated with 35.196.37.91 in the above example never showed up in conntrack (but did in netstat). When I telnet outbound to port 587 of my home server, it shows up in netstat but not in conntrack -L. Could that be causing the issue? Why would my server (stock ubuntu 20.04 from OVH) not be showing outbound connections in conntrack?

CpnCrunch

6/25/23, 6:26 PM

Also, looking again at the logs, all the blocked packets seem to be RST or ACKs. I guess the RST isn't a problem, but the ACKs might be.

Doug Smythies

6/25/23, 6:53 PM

does "OVH" mean it is a hosted server? That your connection show in the tracking table is rather fundamental, so I don't know why yours don't.

CpnCrunch

6/25/23, 7:18 PM

No, it's a dedicated server with OVH. I tried on a cloud server with OVH using ubuntu 20.04, and conntrack -L is showing no entries at all, inbound or outbound. I had to install conntrack and conntrackd. But even with no connection tracking, I can set up ufw. So does ufw have its own connection tracking? The whole thing is very confusing and poorly documented in this respect, and I'm very leery of running it on my prod server until I understand it properly.

Doug Smythies

6/25/23, 10:36 PM

well, there isn't a conntrack table unless iptables uses it. Note that UFW is just a front end for iptables. The way to understand this stuff is to use iptables to look at packet flow and counters and such in the rule set that UFW creates. Myself I hate UFW and the complicated hard to read iptables rule set it creates. My test server has no iptables rule set at all, and the conntrack table shows as empty, evn though I have multiple SSH sessions running.

CpnCrunch

6/25/23, 10:42 PM

Yeah, I used to use raw iptables on centos without any issues. ufw just seems a much more reasonable way of managing a firewall than crafting iptables by hand.

Score:0

Ubuntu

Level9

6/25/23, 7:34 AM

If you look closely you can see that your dropped traffic is incoming traffic DST=1.2.3.4 DST stands for destination and as you stated 1.2.3.4 is your IP.

According to the output of ufw status The traffic has the source-port 443 (SPT) not DPT (which would have been allowed) and thus it is being dropped by ufw.

My guess is that you made a mistake somewhere (probably related to network address translation) that has the effect that the firewall is not stateful.

0 + 0

CpnCrunch

6/25/23, 5:01 PM

That is a response from the remote server (so SRC is the remote server) but the connection is definitely outbound from our server, and is not being tracked by conntrack. I've edited my answer to show the exact commands I used to set up ufw. There are no commands to make it stateful or not, so I don't see how that is my mistake, but please correct me if I'm wrong. The problem seems to be conntrack not tracking outbound connections. Why is that?

Score:0

Ubuntu

CpnCrunch

6/27/23, 5:24 PM

Upon further investigation it turned out that the blocked packets only happened over a period of under 1 second while ufw was being enabled. The "ufw enable" command is nowhere near atomic...it is a python script that interacts with iptables. You might assume that it just does one or two iptables commands, but that is incorrect. Running an strace on "ufw enable" shows that it actually execs the iptables or ip6tables command a total of 358 times in my case:

strace -f -tt ufw --force enable > /tmp/a 2>&1
root@testing:/home/ubuntu# grep exec /tmp/a|wc -l
358
root@testing:/home/ubuntu# grep exec /tmp/a|grep iptables|wc -l
135
root@testing:/home/ubuntu# grep exec /tmp/a|grep ip6tables|wc -l
134

Example:

[pid  1959] 17:13:34.241806 execve("/usr/sbin/ip6tables", ["/usr/sbin/ip6tables", "-I", "ufw6-user-limit", "-m", "limit", "--limit", "3/minute", "-j", "LOG", "--log-prefix", "[UFW LIMIT BLOCK] "], 0x7ffd5e6d3d10 /* 18 vars */ <unfinished ...>

So, the upshot of this is that enabling ufw can temporarily screw up any existing connections that are transmitting or receiving data during the period of time it takes to enable ufw, so be careful enabling ufw on a live server.

0 + 0

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: UFW blocking existing outbound connections when enabled

TH: UFW บล็อกการเชื่อมต่อขาออกที่มีอยู่เมื่อเปิดใช้งาน

RO: UFW blochează conexiunile de ieșire existente atunci când este activat

RU: UFW блокирует существующие исходящие соединения, когда включено

VI: UFW chặn các kết nối ra bên ngoài hiện có khi được bật

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.