Score:1

FreeBSD 13 PF blocking jail traffic

id flag

After upgrading my FreeBSD system from 12.2 to 13.0-p3 PF is blocking all traffic to my jails. When disabling PF everything works fine (except from being unprotected ;))

I tried to figure out what rule is blocking this traffic by setting 'block in log all', but apart from some obvious multicast stuff nothing comes up that could explain why this traffic is blocked.

Again, before everything worked perfectly fine under version 12.2. I did find some articles about v13 now filtering over VLAN instead of lo0 but I don't use any VLANs.

In what direction should I seek further?

Update 2021-07-15:

For clarity sake: heres my pf_rules file:

set block-policy return
set optimization aggressive
set skip on { lo0, lo1, lo2, lo3, lo4, lo5 }
ext_if=hn0
ext_address="{ 192.x.x.x, 2001:981:x.x::x }"
ext_services = "{ ssh, http, https, smtp, smtps }"
tcp_services = "{ ftp, ssh, domain, ntp, www, smtp, smtps, submission, http, https,nfs}"
udp_services = "{ domain, ntp, nfs }"
icmp6_types="{ 2, 128 }" # packet too big, echo request (ping6)
icmp6_types_ext_if="{ 128, 133, 134, 135, 136, 137 }"
jail_net = "192.168.1.0/24"
jail_services = "{ mysql, http, smtp, 587, 3000 }"
table <sshguard> persist
scrub in all
nat pass on $ext_if from $jail_net to any -> $ext_address
block in log on $ext_if proto tcp from <sshguard> to any port ssh label "ssh bruteforce"
block in log all
pass in quick from <pf_whitelist> flags S/SA synproxy state
pass out on $ext_if inet6 proto icmp6 all icmp6-type echoreq keep state
pass out on $ext_if inet proto udp to port 33433:33626
pass out on $ext_if inet6 proto udp to port 33433:33626
pass in on $ext_if inet6 proto ipv6-icmp icmp6-type $icmp6_types keep state
pass in on $ext_if inet6 proto ipv6-icmp from any to { ($ext_if ), ff02::1/16 } icmp6-type $i
cmp6_types_ext_if keep state
pass in on $ext_if proto tcp from any to $ext_address port $ext_services keep state
pass in on $ext_if inet6 proto tcp from any to $ext_address port $ext_services keep state
pass out on $ext_if inet proto tcp to any port $tcp_services keep state
pass out on $ext_if inet6 proto tcp to any port $tcp_services keep state
pass out on $ext_if inet6 proto udp to any port $udp_services
pass proto udp to any port $udp_services keep state
pass in proto tcp from any to $jail_net port $jail_services keep state
pass out proto tcp from $jail_net to any port $jail_services keep state
pass inet proto icmp from any to any

This has worked for many years until FreeBSD 13

drookie avatar
za flag
`kldload pflog && ifconfig pflog0 up && tcpdump -netti pflog0` ? And update your question with the results.
GTeley avatar
id flag
Like I said, I did check the logs (with tcpdump) but apart from some UDP and multicast stuff, nothing comes up: 1626265392.763303 rule 1/0(match): block in on hn0: 192.168.178.56.138 > 192.168.178.255.138: NBT UDP PACKET(138) 1626265398.905896 rule 1/0(match): block in on hn0: fe80::98af:7b38:affd:bc9f > ff02::16: HBH ICMP6, multicast listener report v2, 4 group record(s), length 88
br flag
I had to remove "synproxy" flag.
Score:0
br flag

Quoting from TCP SYN Proxy:

| The SYN proxy will not work if PF is running on a bridge(4).

Remove the flag synproxy

pass in quick from <pf_whitelist> flags S/SA synproxy state

Try instead

pass in quick from <pf_whitelist> flags S/SA keep state

If this works you can use it for jails only and keep synproxy for others, e.g.

pass in quick from <pf_whitelist> to $jail_net flags S/SA keep state
pass in quick from <pf_whitelist> flags S/SA synproxy state
GTeley avatar
id flag
Hello Vladimir, Thanks for your contribution. Unfortunately it doesn't seem te help. Also: I assume there is more to it. Creating a new jail (for test purposes) didn't work because ezjail-admin didn't manage to create links to /bin, /sbin, etc. So I have to delve into this deeper to find out what's more going wrong under version 13. Maybe they should have skippen 13 and go directly to 14 ;)
GTeley avatar
id flag
Yes, there is definitely something wrong with FreeBSD 13 and jails. Even within a jail (the ones that came over from 12.2) I no longer can access the services over the 'external' address they are listening on. So if a local HTTP service is running on port 3000, on a 12.2 jail I can access it with e.g. 'curl -v 192.168.1.101:3000' On a system upgraded to v13 this no longer works. This (partly?) explains why there are no blocked access attempts in de pf_log.
br flag
Sorry to hear that. FWIW, I use the [Ansible role](https://github.com/vbotka/ansible-freebsd-jail). I only had to remove the "synproxy" flag when I moved from 12 to 13.
GTeley avatar
id flag
I installed an old IBM x3400 M3 server with FreeBSD 13 from scratch and created a test_jail, installed some services and everything works fine. So it must have something to do with the upgrade process. As everything I can see is identical in setup and configuration (apart form one being a virtual machine) it's very difficult to tell what has gone wrong
br flag
I see. Would you like to close this Q&A?
Score:0
id flag

Apparantly this had nothing to do with PF but something with the upgrade process from version 12.2 to 13.0. Installing version 13 on a physical server and creating a jail with some services worked fine. So it could have something to do with the upgrade on a virtual server, be it Hyper-V or something else. I still don't know as everything in settings and config looks the same (most part I copied from the upgraded virtual server where it doesn't work)

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.