Join Log in
Score:0
ray an avatar Server

calico networking: pod to pod communication not working

in flag
ray an

ENVIRONMENT:

I am running RKE2 Rancher's Kubernetes distribution. (docs)

I have a single node cluster.
CNI: Calico; Calico is using iptables backend
iptables version: iptables v1.8.7 (legacy)
firewalld version: 0.9.3 (uses nftables by default)

Issue: I have a Hashicorp vault and Kubernetes cronjob running responsible for unsealing and initializing the vault on K8s cluster.

Because of some issue in network, the job is unable to communicate with the vault service running:

Error log from the job

OSError: [ErrNo 113] No route to host

Vault pod ip:

vault-0     10.42.108.2

ip route output

default via 10.225.247.129 dev eth0 proto dhcp 
10.42.108.1 dev cali232527ff5fc scope link 
10.42.108.2 dev calib56e549a1b4 scope link 
10.42.108.3 dev cali7b4b3947ed0 scope link 
10.42.108.4 dev cali480aa60fbbe scope link 
10.42.108.6 dev cali46be8d885e0 scope link 
10.42.108.7 dev cali0c60e5d7209 scope link 
10.42.108.8 dev cali518c430520b scope link 
10.42.108.9 dev cali505dfc32142 scope link 
10.42.108.59 dev cali3ab04dcea6f scope link 
10.42.108.63 dev cali5bcf89d2f73 scope link 
10.225.247.128/25 dev eth0 proto kernel scope link src 10.225.247.171

So, veth calib56e549a1b4 entry corresponding to the pod IP is present in the table

I found few related docs on internet and followed the suggestions but nothing sees to be working. ex: https://github.com/projectcalico/calico/issues/2322 I also tried changing firewalld back end to iptable, still no luck

Calico pods are all working fine.
I am unable to figure out why communication is failing. Any pointers or suggestions to debug this situation.

Feel free to ask if you need more info. Thanks

205
0 + 1
SYN avatar
hk flag
SYN
As a wild guess: your Vault has a readinessProbe, which would fail until the unsealing process is done? Service would not be reachable? Have you tried to tcpdump? Have you check Services / Endpoints? Can you share you Vault deployment configuration? Do you have any other SDN-related issues, or is this just Vault?
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.