Join Log in
Score:0
Jason avatar Server

ARP Connectivity Issues Between OpenMediaVault Server and BSD Jails

cn flag
Jason

On my network I have two servers.

Server1 is running TrueNas(BSD) with multiple applications running in iocage jails. It's connected to the network with a 3-nic LAGG.

Server2 is an OpenMediaVault (Debian) installation with multiple applications that run in docker containers connected with a single physical nic.

Both servers are connected to the same TP-Link managed switch.

I am trying to get a jail on Server1 which runs borgbackup to make http connections to the HealthChecks container on Server2. From the borgbackup jail's console I am unable to ping the IP address of Server2 or make curl requests to the docker container, even though it is up and works from everywhere else on the subnet, including from Server1 outside of the jail. If I then go to the console of Server2 and ping the IP of the jail, after missing (timeouts) 8-10 pings all communication is possible temporarily, which leads me to believe this is an ARP issue, but I'm unsure how to solve it.

Other possibly unrelated weirdness:

  • The network is a /21 sized network, and both machines are in different /24 subnets (all nodes are configured as /21 so this shouldn't actually matter).
  • I've noticed is that jails have never been able to get IP addresses from the DHCP server (I usually do DHCP reservation for all IP assignment on the network) and I've had to manually assign IPs for all of my jails on Server1.
  • When the connection is not working pings from Server2 to the jail IP timeout. Pings from the jail to Server2 return ping: sendto: Host is down.

For the community bot: The main question is why does this communication channel fail after what I assume is the arp cache expiration, and how do I fix it?

EDIT: Disabling VNET on the jails seems to resolve the issue, but I was hoping to be able to keep VNET enabled.

41
0 + 0
arp
bsd
Ron Maupin avatar
us flag
Ron Maupin
The ping failure message should give you a clue if it is an ARP problem. Does it time out, or a host not found, or some other message?
0
Reply
Jason avatar
cn flag
Jason
They are timeouts from Server2 to the jail. Pinging from the jail to Server2 while it's not working gives ping: sendto: Host is down
0
Reply
Ron Maupin avatar
us flag
Ron Maupin
Timeout means you have an ARP entry. A host not found error means you do not have an ARP entry.
0
Reply
Ron Maupin avatar
us flag
Ron Maupin
If the jails cannot get DHCP nor ARP information, then broadcast is broken for the jails It sounds like they are not on the same network as the DHCP server or other server, or at lest they do not believe it (different mask). Your mention of `/24` networks actually means nothing except that you may have something with the wrong mask. All masks on the servers and jails needs to be the same (`255.255.248.0` for `/21`).
0
Reply
Jason avatar
cn flag
Jason
the netmask was the first thing I checked, everyone is in the same /21 or 255.255.248.0. The DHCP server, the jail, Server1's LAGG, and Server2. Mostly I see this stuff on the LAGG adapter, but not the other adapter (on a different vlan) on the server.
0
Reply
Ron Maupin avatar
us flag
Ron Maupin
You have something breaking broadcast from the jails. For example, broadcasts cannot be routed, so they will not cross a layer-3 boundary.
0
Reply
Jason avatar
cn flag
Jason
Turning off VNET in the jail configuration and attaching directly to the lagg adapter seems to allow the communication to work.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.