I had to update Kubernetes which broke some stuff. Load Balancers not accepting traffic. Cert-manager and Ingress-controller pods red so not running. I had to update cert-manager and ingress. Both the cert-manager pod and the ingress pod is not green and running. This managed to get one load balancer pool back online.
kubectl get all -n ingress-nginx
NAME READY STATUS RESTARTS AGE
pod/ingress-nginx-admission-create-wdl65 0/1 Completed 0 23h
pod/ingress-nginx-admission-patch-wqdhl 0/1 Completed 1 23h
pod/ingress-nginx-controller-7575567f98-sqcx6 1/1 Running 0 3m34s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/ingress-nginx-controller LoadBalancer 10.245.11.188 167.99.27.7 80:30719/TCP,443:31445/TCP 23h
service/ingress-nginx-controller-admission ClusterIP 10.245.17.44 <none> 443/TCP 23h
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/ingress-nginx-controller 1/1 1 1 23h
NAME DESIRED CURRENT READY AGE
replicaset.apps/ingress-nginx-controller-5cc4f7674 0 0 0 23h
replicaset.apps/ingress-nginx-controller-7575567f98 1 1 1 3m34s
replicaset.apps/ingress-nginx-controller-85fb844f76 0 0 0 23h
NAME COMPLETIONS DURATION AGE
job.batch/ingress-nginx-admission-create 1/1 9s 23h
job.batch/ingress-nginx-admission-patch 1/1 10s 23h
helm list -a
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
device-locator default 1 2021-10-10 15:57:37.195218278 -0700 -0700 deployed device-locator-0.3.0 latest
emulation-lab default 4 2021-08-21 05:57:42.576446795 +0000 UTC deployed emulation-lab-0.1.0 latest
inlo-api-server default 3 2022-10-17 23:07:40.007303284 +0000 UTC deployed inlo-api-server-0.3.0 latest
inlo-communication-server default 1 2021-10-10 16:00:52.806929875 -0700 -0700 deployed inlo-communication-server-0.1.0 latest
kafka default 1 2021-10-09 22:35:15.584331665 -0700 -0700 deployed kafka-14.0.1 2.8.0
mongo default 1 2021-07-21 17:56:33.186650093 -0700 -0700 deployed mongodb-10.12.6 4.4.5
mosquitto default 10 2021-07-27 22:35:33.814959285 -0700 -0700 deployed mosquitto-0.2.0 1.16.0
nginx-ingress default 1 2022-10-26 12:19:20.128985611 -0400 EDT failed ingress-nginx-4.3.0 1.4.0
web-portal default 32 2022-10-11 03:40:01.062028765 +0000 UTC deployed web-portal-0.1.0 latest
ws-api-server default 1 2021-10-10 15:57:29.095758102 -0700 -0700 deployed ws-api-server-0.1.0 latest
I notice Helm shows deployment failed. I tried to upgrade/install ingress-nginx again with Helm.
helm install nginx-ingress ingress-nginx/ingress-nginx --set controller.publishService.enabled=true --set controller.service.customPorts=8883 --debug
install.go:192: [debug] Original chart version: ""
install.go:209: [debug] CHART PATH: /home/osboxes/.cache/helm/repository/ingress-nginx-4.3.0.tgz
client.go:310: [debug] Starting delete for "nginx-ingress-ingress-nginx-admission" ServiceAccount
client.go:128: [debug] creating 1 resource(s)
client.go:310: [debug] Starting delete for "nginx-ingress-ingress-nginx-admission" ClusterRole
client.go:339: [debug] clusterroles.rbac.authorization.k8s.io "nginx-ingress-ingress-nginx-admission" not found
client.go:128: [debug] creating 1 resource(s)
client.go:310: [debug] Starting delete for "nginx-ingress-ingress-nginx-admission" ClusterRoleBinding
client.go:339: [debug] clusterrolebindings.rbac.authorization.k8s.io "nginx-ingress-ingress-nginx-admission" not found
client.go:128: [debug] creating 1 resource(s)
client.go:310: [debug] Starting delete for "nginx-ingress-ingress-nginx-admission" Role
client.go:128: [debug] creating 1 resource(s)
client.go:310: [debug] Starting delete for "nginx-ingress-ingress-nginx-admission" RoleBinding
client.go:128: [debug] creating 1 resource(s)
client.go:310: [debug] Starting delete for "nginx-ingress-ingress-nginx-admission-create" Job
client.go:128: [debug] creating 1 resource(s)
client.go:540: [debug] Watching for changes to Job nginx-ingress-ingress-nginx-admission-create with timeout of 5m0s
client.go:568: [debug] Add/Modify event for nginx-ingress-ingress-nginx-admission-create: ADDED
client.go:607: [debug] nginx-ingress-ingress-nginx-admission-create: Jobs active: 0, jobs failed: 0, jobs succeeded: 0
Error: INSTALLATION FAILED: failed pre-install: timed out waiting for the condition
helm.go:84: [debug] failed pre-install: timed out waiting for the condition
INSTALLATION FAILED
main.newInstallCmd.func2
This seems to have issue with ingress admission create
