I've configured the bridging and the tap interface on freebsd 13 because I want to boot and use a bhyve VM based on Ubuntu 21.04 and Windows 10,but unfortunately,after several readings and help requests on :
reddit : https://www.reddit.com/r/Ubuntu/comments/p38ru1/ubuntu_2104_cant_connect_to_internet_when_it_is/
the freebsd forums :
https://forums.freebsd.org/threads/how-to-duplicate-the-tap0-interface-in-a-new-tap1-network-interface-that-it-is-not-used.81671/page-2#post-527355
https://forums.freebsd.org/threads/ubuntu-budgie-emulated-with-bhyve-doesnt-connect-to-internet-because-the-error-network-unreachable-n-2.81340/#post-524303
https://forums.freebsd.org/threads/im-tryng-to-emulate-ubuntu-budgie-with-bhyve-but-it-doesnt-connect-to-internet-because-the-error-network-unreachable.80916/#post-517845
askubuntu : https://askubuntu.com/questions/1357670/ubuntu-21-04-cant-connect-to-internet-when-it-is-emulated-with-bhyve-on-my-free
unix stackexchange : https://unix.stackexchange.com/questions/664466/ubuntu-21-04-cant-connect-to-internet-when-it-is-emulated-with-bhyve-on-my-free?noredirect=1#comment1250440_664466
I haven't been able to understand why Ubuntu and Windows 10 VM can't connect to the internet,while if I use vm-bhyve,it configures the network in some different way and the vm can connect to the internet. Since I'm not a coder,I'm not able to look inside the vm-bhyve code and I'm not able to understand what's the secret key. Below I want to give you some useful information for understanding how I have configured the bridge and the tap interface on FreeBSD and on Ubuntu.
/etc/resolv.conf
search homenet.telecomitalia.it
nameserver 127.0.0.1
options edns0
my /etc/rc.conf is like this :
hostname="marietto"
keymap="it.kbd"
ifconfig_em0="DHCP"
local_unbound_enable="YES"
sshd_enable="YES"
powerd_enable="YES"
ntpdate_enable="YES"
ntpd_enable="YES"
ntpd_sync_on_start="YES"
dumpdev="NO"
dbus_enable="YES"
slim_enable="NO"
libvirt_enable="YES"
linux_enable="YES"
linux_mounts_enable="YES"
gateway_enable="YES"
kld_list="fusefs"
lightdm_enable="NO"
sendmail_enable="NO"
microcode_update_enable="YES"
fsck_y_enable="YES"
tcp_drop_synfin="NO"
telnet_enable="NO"
cupsd_enable="NO"
portmap_enable="NO"
rlogin_enable="NO"
inetd_enable="NO"
webcamd_enable="NO"
samba_enable="NO"
lpd_enable="NO"
winbindd_enable="NO"
nfs_server_enable="NO"
nfs_client_enable="NO"
vm_enable="YES"
vm_dir="/mnt/da1p1/vms/os/"
my /boot/loader.conf is like this :
vmm_load="YES"
nmdm_load="YES"
if_tap_load="YES"
if_bridge_load="YES"
fusefs_load="YES"
linprocfs_load="YES"
linsysfs_load="YES"
tmpfs_load="YES"
verbose_loading="YES"
vboxdrv_load="YES"
console=vga,com1
boot_multicons="YES"
boot_serial="YES"
console="comconsole,vidconsole"
my file /etc/sysctl.conf is like this :
net.link.tap.up_on_open=1
net.inet.ip.forwarding=1
I've read here to get some orientation about what to do : https://wiki.freebsd.org/bhyve/UEFI,specially specially in this section :
kldload vmm
ifconfig tap0 create up
ifconfig bridge0 create up
ifconfig bridge0 addm em0
As a first attempt,with the suggestions of Vasily Postnicov,I tried to configure the network interfaces of freebsd and of linux in a different way,to be able to give the internet connection to ubuntu emulated with bhyve. This is what I did :
root@marietto:/home/marietto # ifconfig em0 delete
root@marietto:/home/marietto # dhclient bridge0
DHCPDISCOVER on bridge0 to 255.255.255.255 port 67 interval 4
DHCPOFFER from 192.168.1.1
DHCPREQUEST on bridge0 to 255.255.255.255 port 67
DHCPACK from 192.168.1.1
bound to 192.168.1.2 -- renewal in 10800 seconds.
root@marietto:/home/marietto # ifconfig tap0 create up
root@marietto:/home/marietto # ifconfig bridge0 addm tap0
root@marietto:/usr/home/marietto # ifconfig
em0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4810099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWFILTER,NOMAP>
ether e0:d5:5e:e2:1f:22
inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 58:9c:fc:10:ff:b7
inet 192.168.1.2 netmask 0xffffff00 broadcast 192.168.1.255
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 5 priority 128 path cost 2000000
member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 2000000
groups: bridge
nd6 options=9<PERFORMNUD,IFDISABLED>
vm-public: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether b6:74:ff:a8:4d:4e
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
groups: bridge vm-switch viid-4c918@
nd6 options=9<PERFORMNUD,IFDISABLED>
tap0: flags=8903<UP,BROADCAST,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
ether 58:9c:fc:10:ff:8e
groups: tap
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Unfortunately as soon as the vm boots,the em0 interface on freebsd gets again the old IP number : 192.168.1.6 and I lose the connection to the net on FreeBSD.
As a further try this is what I did : (take in consideration that I'm trying to make a parallel network configuration on Freebsd,since I want to keep vm-bhyve installed. Anyway,I've removed vm-bhyve several times and I tried to configure the bridge network for bhyve,but even in this case it didn't work. In short terms,it worked only the first time,but I haven't been able to repeat succesfully the steps anymore. It's working only if I let vm-bhyve configure the network by itself. This is the initial state of the network / bridge configuration on my FreeBSD with vm-bhyve installed but without starting a virtual machine.
marietto@marietto:~ $ ifconfig
em0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=481009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWFILTER,NOMAP>
ether e0:d5:5e:e2:1f:22
inet 192.168.1.6 netmask 0xffffff00 broadcast 192.168.1.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vm-public: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 0a:77:93:94:d5:77
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 20000
groups: bridge vm-switch viid-4c918@
nd6 options=9<PERFORMNUD,IFDISABLED>
Since I already have a bridged interface (vm-public) it does not make any sense to create another one (bridge0),so on the terminal I do :
root@marietto:/home/marietto # kldload vmm
kldload: can't load vmm: module already loaded or in kernel
root@marietto:/home/marietto # ifconfig tap0 create up
root@marietto:/home/marietto # ifconfig vm-public addm tap0
so,this is how appear the network configuration on FreeBSD after having configured it like above :
root@marietto:/home/marietto # ifconfig
em0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4810099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWFILTER,NOMAP>
ether e0:d5:5e:e2:1f:22
inet 192.168.1.6 netmask 0xffffff00 broadcast 192.168.1.255
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vm-public: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 0a:77:93:94:d5:77
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 4 priority 128 path cost 2000000
member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 20000
groups: bridge vm-switch viid-4c918@
nd6 options=9<PERFORMNUD,IFDISABLED>
tap0: flags=8903<UP,BROADCAST,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
ether 58:9c:fc:10:ff:8e
groups: tap
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Then,I launch the bhyve VM with this commands :
bhyve -c 4 -m 8G -w -H \
-s 0,hostbridge \
-s 4,virtio-blk,/mnt/da1p1/vms/os/ubuntu-budgie-gpu/ubuntu-2104-gpu.img \
-s 5,virtio-net,tap0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1440,h=900,wait \
-s 30,xhci,tablet \
-s 31,lpc -l com1,stdio \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
-s 20,hda,play=/dev/dsp8,rec=/dev/dsp8 \
vm0
As I said Ubuntu VM can't connect to the internet. Below I want to show you how I have configured Ubuntu,instead. On Ubuntu my /etc/resolv.conf file is like this :
nameserver 127.0.0.53
options edns0 trust-ad
search homenet.telecomitalia.it
In addition I've created a netplan file inside the folder /etc/netplan,with this content inside :
network
version: 2
renderer: NetworkManager
ethernets:
enp0s5:
addresses:
- 192.168.1.7/24
gateway4: 192.168.1.1
nameservers:
search : [homenet.telecomitalia.it]
addresses: 192.168.1.1
netplan apply and this is what happens :
the network interface enp0s5 gets the IP number 192.168.1.7
ping 192.168.1.7 : OK
ping 192.168.1.6 (FreeBSD IP) = destination host unreachable
ping 192.168.1.1 : destination host unreachable
PING google.it (172.217.21.67) : host unreachable
tcpdump :
listening on enp0s5,link-type EN10MB (Ethernet),capture size 262144 bytes
20:58:04:688913 ARP, request who-has 192.168.1.1 tell 192.168.1.7,length 28
route -n
destination gateway genmask flags metric ref use iface
0.0.0.0 192.168.1.1 0.0.0.0 UG 100 0 0 enp0s5
169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 enp0s5
192.168.1.0 0.0.0.0 255.255.255.0 U 100 0 0 enp0s5
ip route list
default via 192.168.1.1 dev enp0s5 proto static metric 100
169.254.0.0/16 dev enp0s5 scope link metric 100
192.168.1.0/24 dev enp0s5 proto kernel scope link src 192.168.1.7 metric 100
ping google.it = temporary error in name resolution
firewall on ubuntu = not installed
firewall on router = down
instead,from freebsd to ubuntu :
ping 192.168.1.7 : host is down
netstar -r
routing tables
internet:
destination gateway flags Netif Expire
default modemtim UGS em0
localhost link#2 UH lo0
192.168.1.0/24 link#1 U em0
192.168.1.6 link#1 UHS lo0
I forgot. This is what happens if I remove the netplan conf file and I let ubuntu choose the IP via DHCP :
/etc/netplan.conf
network:
version: 2
renderer: networkd
ethernets:
enp3s0:
dhcp4: true
ping PING google.it (172.217.21.67) : network unreachable. what else ?
PS : Since I'm not a real system administrator,but I do it as an hobby,don't give me generic suggestions because probably I will not understant what to do. If you want to be a kind person,try to be exhaustive and precise. You will help me a lot to understand how to fix the problem. thanks.
