I've installed kubernetes master and one node v 1.20. I deployed nginx with
kubectl run nginxpod --image=nginx
$ kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginxpod 1/1 Running 0 19s 192.168.2.195 xps15-9560 <none> <none>
On master when I curl on master it times out:
$ curl 192.168.2.195
curl: (7) Failed to connect to 192.168.2.195 port 80: Connection timed out
On the node it works. I've tried from other hosts on my network and they timeout too. Why can I only connect from the node the pod is actually running on?
----Edit----
The calico-nodes are running but they are not ready. I don't know what this means:
$ kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
default nginxpod 1/1 Running 0 64m
kube-system calico-kube-controllers-5f6cfd688c-wk5jp 1/1 Running 0 69m
kube-system calico-node-t47kf 0/1 Running 0 45m
kube-system calico-node-vqj6m 0/1 Running 0 68m
kube-system calico-node-wzwzb 0/1 Running 0 69m
kube-system coredns-74ff55c5b-mb2vj 1/1 Running 0 69m
kube-system coredns-74ff55c5b-pvsgz 1/1 Running 0 69m
kube-system etcd-ubuntu-18-extssd 1/1 Running 0 69m
kube-system kube-apiserver-ubuntu-18-extssd 1/1 Running 0 69m
kube-system kube-controller-manager-ubuntu-18-extssd 1/1 Running 0 69m
kube-system kube-proxy-5fq9b 1/1 Running 0 68m
kube-system kube-proxy-bxhfm 1/1 Running 0 69m
kube-system kube-proxy-pp9sb 1/1 Running 0 45m
kube-system kube-scheduler-ubuntu-18-extssd 1/1 Running 0 69m
--------Edit 2------------------
Calico nodes NotFound:
$ kubectl describe pod calico-node-t47kf
Error from server (NotFound): pods "calico-node-t47kf" not found
$ kubectl describe pod calico-node-vqj6m
Error from server (NotFound): pods "calico-node-vqj6m" not found
$ kubectl describe pod calico-node-*****
Error from server (NotFound): pods "calico-node-*****" not found
----Edit 3----
Output for describe pod calico-node-t47kf:
$ kubectl -n kube-system describe pod calico-node-t47kf
Name: calico-node-t47kf
Namespace: kube-system
Priority: 2000001000
Priority Class Name: system-node-critical
Node: xps15-9560/192.168.0.71
Start Time: Sun, 05 Sep 2021 16:51:37 -0600
Labels: controller-revision-hash=b8998dcb
k8s-app=calico-node
pod-template-generation=1
Annotations: <none>
Status: Running
IP: 192.168.0.71
IPs:
IP: 192.168.0.71
Controlled By: DaemonSet/calico-node
Init Containers:
upgrade-ipam:
Container ID: docker://3d393316548badf75bb2c2ad881ffd7a4d2c37a1762d84ec973715c8a398072e
Image: docker.io/calico/cni:v3.20.0
Image ID: docker-pullable://calico/cni@sha256:9906e2cca8006e1fe9fc3f358a3a06da6253afdd6fad05d594e884e8298ffe1d
Port: <none>
Host Port: <none>
Command:
/opt/cni/bin/calico-ipam
-upgrade
State: Terminated
Reason: Completed
Exit Code: 0
Started: Sun, 05 Sep 2021 16:51:38 -0600
Finished: Sun, 05 Sep 2021 16:51:38 -0600
Ready: True
Restart Count: 0
Environment Variables from:
kubernetes-services-endpoint ConfigMap Optional: true
Environment:
KUBERNETES_NODE_NAME: (v1:spec.nodeName)
CALICO_NETWORKING_BACKEND: <set to the key 'calico_backend' of config map 'calico-config'> Optional: false
Mounts:
/host/opt/cni/bin from cni-bin-dir (rw)
/var/lib/cni/networks from host-local-net-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-qfrcw (ro)
install-cni:
Container ID: docker://9e3b23f12657fe343117ec0cf54e104a8eb69c6133fe2dd10c1aabbc9260189f
Image: docker.io/calico/cni:v3.20.0
Image ID: docker-pullable://calico/cni@sha256:9906e2cca8006e1fe9fc3f358a3a06da6253afdd6fad05d594e884e8298ffe1d
Port: <none>
Host Port: <none>
Command:
/opt/cni/bin/install
State: Terminated
Reason: Completed
Exit Code: 0
Started: Sun, 05 Sep 2021 16:51:39 -0600
Finished: Sun, 05 Sep 2021 16:51:40 -0600
Ready: True
Restart Count: 0
Environment Variables from:
kubernetes-services-endpoint ConfigMap Optional: true
Environment:
CNI_CONF_NAME: 10-calico.conflist
CNI_NETWORK_CONFIG: <set to the key 'cni_network_config' of config map 'calico-config'> Optional: false
KUBERNETES_NODE_NAME: (v1:spec.nodeName)
CNI_MTU: <set to the key 'veth_mtu' of config map 'calico-config'> Optional: false
SLEEP: false
Mounts:
/host/etc/cni/net.d from cni-net-dir (rw)
/host/opt/cni/bin from cni-bin-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-qfrcw (ro)
flexvol-driver:
Container ID: docker://090008276cf33c2fd64aa141405e90feff00fc70f55769372655e45bf2e2dc92
Image: docker.io/calico/pod2daemon-flexvol:v3.20.0
Image ID: docker-pullable://calico/pod2daemon-flexvol@sha256:c17e3e9871682bed00bfd33f8d6f00db1d1a126034a25bf5380355978e0c548d
Port: <none>
Host Port: <none>
State: Terminated
Reason: Completed
Exit Code: 0
Started: Sun, 05 Sep 2021 16:51:40 -0600
Finished: Sun, 05 Sep 2021 16:51:40 -0600
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/host/driver from flexvol-driver-host (rw)
/var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-qfrcw (ro)
Containers:
calico-node:
Container ID: docker://2c1456bb2f346564f88b3e8deba3baef1669f0aa61aa0e654d52c915eaf462cf
Image: docker.io/calico/node:v3.20.0
Image ID: docker-pullable://calico/node@sha256:7f9aa7e31fbcea7be64b153f8bcfd494de023679ec10d851a05667f0adb42650
Port: <none>
Host Port: <none>
State: Running
Started: Sun, 05 Sep 2021 16:51:41 -0600
Ready: False
Restart Count: 0
Requests:
cpu: 250m
Liveness: exec [/bin/calico-node -felix-live -bird-live] delay=10s timeout=10s period=10s #success=1 #failure=6
Readiness: exec [/bin/calico-node -felix-ready -bird-ready] delay=0s timeout=10s period=10s #success=1 #failure=3
Environment Variables from:
kubernetes-services-endpoint ConfigMap Optional: true
Environment:
DATASTORE_TYPE: kubernetes
WAIT_FOR_DATASTORE: true
NODENAME: (v1:spec.nodeName)
CALICO_NETWORKING_BACKEND: <set to the key 'calico_backend' of config map 'calico-config'> Optional: false
CLUSTER_TYPE: k8s,bgp
IP: autodetect
CALICO_IPV4POOL_IPIP: Always
CALICO_IPV4POOL_VXLAN: Never
FELIX_IPINIPMTU: <set to the key 'veth_mtu' of config map 'calico-config'> Optional: false
FELIX_VXLANMTU: <set to the key 'veth_mtu' of config map 'calico-config'> Optional: false
FELIX_WIREGUARDMTU: <set to the key 'veth_mtu' of config map 'calico-config'> Optional: false
CALICO_DISABLE_FILE_LOGGING: true
FELIX_DEFAULTENDPOINTTOHOSTACTION: ACCEPT
FELIX_IPV6SUPPORT: false
FELIX_HEALTHENABLED: true
Mounts:
/host/etc/cni/net.d from cni-net-dir (rw)
/lib/modules from lib-modules (ro)
/run/xtables.lock from xtables-lock (rw)
/sys/fs/ from sysfs (rw)
/var/lib/calico from var-lib-calico (rw)
/var/log/calico/cni from cni-log-dir (ro)
/var/run/calico from var-run-calico (rw)
/var/run/nodeagent from policysync (rw)
/var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-qfrcw (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
lib-modules:
Type: HostPath (bare host directory volume)
Path: /lib/modules
HostPathType:
var-run-calico:
Type: HostPath (bare host directory volume)
Path: /var/run/calico
HostPathType:
var-lib-calico:
Type: HostPath (bare host directory volume)
Path: /var/lib/calico
HostPathType:
xtables-lock:
Type: HostPath (bare host directory volume)
Path: /run/xtables.lock
HostPathType: FileOrCreate
sysfs:
Type: HostPath (bare host directory volume)
Path: /sys/fs/
HostPathType: DirectoryOrCreate
cni-bin-dir:
Type: HostPath (bare host directory volume)
Path: /opt/cni/bin
HostPathType:
cni-net-dir:
Type: HostPath (bare host directory volume)
Path: /etc/cni/net.d
HostPathType:
cni-log-dir:
Type: HostPath (bare host directory volume)
Path: /var/log/calico/cni
HostPathType:
host-local-net-dir:
Type: HostPath (bare host directory volume)
Path: /var/lib/cni/networks
HostPathType:
policysync:
Type: HostPath (bare host directory volume)
Path: /var/run/nodeagent
HostPathType: DirectoryOrCreate
flexvol-driver-host:
Type: HostPath (bare host directory volume)
Path: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds
HostPathType: DirectoryOrCreate
calico-node-token-qfrcw:
Type: Secret (a volume populated by a Secret)
SecretName: calico-node-token-qfrcw
Optional: false
QoS Class: Burstable
Node-Selectors: kubernetes.io/os=linux
Tolerations: :NoSchedule op=Exists
:NoExecute op=Exists
CriticalAddonsOnly op=Exists
node.kubernetes.io/disk-pressure:NoSchedule op=Exists
node.kubernetes.io/memory-pressure:NoSchedule op=Exists
node.kubernetes.io/network-unavailable:NoSchedule op=Exists
node.kubernetes.io/not-ready:NoExecute op=Exists
node.kubernetes.io/pid-pressure:NoSchedule op=Exists
node.kubernetes.io/unreachable:NoExecute op=Exists
node.kubernetes.io/unschedulable:NoSchedule op=Exists
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Unhealthy 93s (x41181 over 4d18h) kubelet (combined from similar events): Readiness probe failed: 2021-09-10 17:16:42.497 [INFO][1620263] confd/health.go 180: Number of node(s) with BGP peering established = 0
calico/node is not ready: BIRD is not ready: BGP not established with 192.168.55.1,192.168.0.12
------Edit 4-------
Tried deleting and reinstalling calico and that made things worse.
