I have a IPv6 address block of prefix size 64. On my host computer running FreeBSD 13-RELEASE I have three jails:
jls
JID IP Address Hostname Path
1 svcfw /usr/jail/svcfw
2 www /usr/jail/www
3 ifee /usr/jail/ifee
Host network adapter connected to global Internet:
ifconfig vtnet0
vtnet0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4804bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LRO,LINKSTATE,TXCSUM_IPV6>
ether 56:00:03:7a:2c:a1
inet 192.248.184.48 netmask 0xfffffe00 broadcast 192.248.185.255
inet6 fe80::5400:3ff:fe7a:2ca1%vtnet0 prefixlen 64 scopeid 0x1
inet6 2001:19f0:6c01:2033::2 prefixlen 65
media: Ethernet autoselect (10Gbase-T <full-duplex>)
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
Contents of my /etc/rc.conf:
hostname="login"
clear_tmp_enable="YES"
syslogd_flags="-ss"
sendmail_enable="NONE"
static_routes="linklocal"
route_linklocal="-net 169.254.0.0/16 -interface vtnet0"
ifconfig_vtnet0="inet 192.248.184.48 netmask 255.255.254.0"
defaultrouter="192.248.184.1"
ifconfig_vtnet0_ipv6="inet6 2001:19f0:6c01:2033::2 prefixlen 65"
ipv6_default_interface="vtnet0"
# XXX:
# - https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=233283#c4
# - https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=233283#c16
ipv6_defaultrouter="fe80::fc00:3ff:fe7a:2ca1%vtnet0"
cloned_interfaces="bridge0 epair0"
ifconfig_epair0a_ipv6="inet6 2001:19f0:6c01:2033:8000::1 prefixlen 65"
ifconfig_epair0a="inet 10.22.0.1 netmask 255.255.0.0"
# Private network: net5ab50b04a6b66
ifconfig_vtnet1="inet 10.9.96.5 netmask 255.255.240.0 mtu 1450"
#pf_enable="YES"
#pf_rules="/etc/pf.conf"
#pflog_enable="YES"
#pflog_logfile="/var/log/pflog"
sshd_enable="YES"
dumpdev="NO"
zfs_enable="YES"
devmatch_blacklist="virtio_random.ko"
gateway_enable="YES"
ipv6_gateway_enable="YES"
wireguard_enable="YES"
wireguard_interfaces="wg0"
jail_enable="YES"
Networking from host is working as it should:
fetch -6 -o /dev/null http://example.com
/dev/null 1256 B 10 MBps 00s
ping6 -c 4 google.com
PING6(56=40+8+8 bytes) 2001:19f0:6c01:2033::2 --> 2a00:1450:4001:802::200e
16 bytes from 2a00:1450:4001:802::200e, icmp_seq=0 hlim=119 time=0.832 ms
16 bytes from 2a00:1450:4001:802::200e, icmp_seq=1 hlim=119 time=0.793 ms
16 bytes from 2a00:1450:4001:802::200e, icmp_seq=2 hlim=119 time=0.867 ms
16 bytes from 2a00:1450:4001:802::200e, icmp_seq=3 hlim=119 time=0.827 ms
--- google.com ping6 statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.793/0.830/0.867/0.026 ms
Main network adapter of the first jail:
doas jexec svcfw ifconfig bridge0
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 58:9c:fc:10:e2:24
inet6 2001:19f0:6c01:2033:8000::2 prefixlen 65
inet 10.22.0.2 netmask 0xffff0000 broadcast 10.22.255.255
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: epair2a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 5 priority 128 path cost 2000
member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 4 priority 128 path cost 2000
member: epair0b flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 3 priority 128 path cost 2000
groups: bridge
nd6 options=8001<PERFORMNUD,DEFAULTIF>
Ping jail from host works as it should:
ping6 -c 4 2001:19f0:6c01:2033:8000::2
PING6(56=40+8+8 bytes) 2001:19f0:6c01:2033:8000::1 --> 2001:19f0:6c01:2033:8000::2
16 bytes from 2001:19f0:6c01:2033:8000::2, icmp_seq=0 hlim=64 time=0.077 ms
16 bytes from 2001:19f0:6c01:2033:8000::2, icmp_seq=1 hlim=64 time=0.062 ms
16 bytes from 2001:19f0:6c01:2033:8000::2, icmp_seq=2 hlim=64 time=0.080 ms
16 bytes from 2001:19f0:6c01:2033:8000::2, icmp_seq=3 hlim=64 time=0.062 ms
--- 2001:19f0:6c01:2033:8000::2 ping6 statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.062/0.070/0.080/0.008 ms
Ping host from jail works too as it should:
doas jexec svcfw ping6 -c 4 2001:19f0:6c01:2033:8000::1
PING6(56=40+8+8 bytes) 2001:19f0:6c01:2033:8000::2 --> 2001:19f0:6c01:2033:8000::1
16 bytes from 2001:19f0:6c01:2033:8000::1, icmp_seq=0 hlim=64 time=0.068 ms
16 bytes from 2001:19f0:6c01:2033:8000::1, icmp_seq=1 hlim=64 time=0.064 ms
16 bytes from 2001:19f0:6c01:2033:8000::1, icmp_seq=2 hlim=64 time=0.059 ms
16 bytes from 2001:19f0:6c01:2033:8000::1, icmp_seq=3 hlim=64 time=0.060 ms
--- 2001:19f0:6c01:2033:8000::1 ping6 statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.059/0.063/0.068/0.004 ms
Configuration of host-side pair connected to bridge:
ifconfig epair0a
epair0a: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:a8:b4:85:57:0a
inet 10.22.0.1 netmask 0xffff0000 broadcast 10.22.255.255
inet6 fe80::a8:b4ff:fe85:570a%epair0a prefixlen 64 scopeid 0x5
inet6 2001:19f0:6c01:2033:8000::1 prefixlen 65
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
Ordinarily, pf is doing NAT for IPv4. Currently I have disabled pf so that pf is not running while I am trying to get global IPv6 to work for the jails.
The host is configured to route IPv6 traffic, cf /etc/rc.conf that I included earlier in this message.
Here is what the jail says about the route to an IPv6 address on the internet:
doas jexec svcfw route -6 get 2a00:1450:400f:802::200e
route to: 2a00:1450:400f:802::200e
destination: default
mask: default
gateway: 2001:19f0:6c01:2033:8000::1
fib: 0
interface: bridge0
flags: <UP,GATEWAY,DONE,STATIC>
recvpipe sendpipe ssthresh rtt,msec mtu weight expire
0 0 0 0 1500 1 0
Should be sent to bridge0, looks good to me.
Here is what the host tells me about the routes for google.com and for the jail respectively:
route -6 get google.com
route to: fra24s01-in-x0e.1e100.net
destination: default
mask: default
gateway: fe80::fc00:3ff:fe7a:2ca1%vtnet0
fib: 0
interface: vtnet0
flags: <UP,GATEWAY,DONE,STATIC>
recvpipe sendpipe ssthresh rtt,msec mtu weight expire
0 0 0 0 1500 1 0
route -6 get 2001:19f0:6c01:2033::2
route to: 2001:19f0:6c01:2033::2
destination: 2001:19f0:6c01:2033::2
fib: 0
interface: lo0
flags: <UP,HOST,DONE,STATIC,PINNED>
recvpipe sendpipe ssthresh rtt,msec mtu weight expire
0 0 0 0 16384 1 0
Looks mostly good. Dunno why the latter route says the loopback interface lo0 instead of the epair interface epair0a. Is this a problem? Or is something else causing the problem?
The problem being that I cannot reach the public Internet by IPv6 from my jail, even though IPv6 routing on the host is on and the host firewall (pf) is off, and the host can speak with the Internet over IPv6 just fine and I am using addresses from my assigned /64. (Assigning more addresses from the /64 to the host works fine btw.)
When I try to even ping something on the Internet by IPv6 that worked fine from the host, doesn't work from the jail:
doas jexec svcfw ping6 -c 4 2a00:1450:400f:802::200e
PING6(56=40+8+8 bytes) 2001:19f0:6c01:2033:8000::2 --> 2a00:1450:400f:802::200e
--- 2a00:1450:400f:802::200e ping6 statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss
Doesn't work.
What am I missing? What am I doing wrong? Been going on at this for hours.
