Score:0

Application deployed to Kubernetes pod was creating pod successfully in Jenkins-X 2 but fails on Jenkins-X 3

us flag

I've inherited an application I need to deploy inside the cluster as a persistent pod to have access to other resources, but which only runs on demand when a user kubectl execs into the pod. At initialisation I don't need it to do anything at all other than make the pod available for a user to exec into at a later date.

This works fine on our old Jenkins-X 2 cluster, but the new Jenkinx-X 3 version is going nowhere.

When it's deployed the status appears to go through a lifecycle of

Running
Completed
CrashLoopBackOff

However kubectl logs -n <<namespace>> <<podname>> -p shows no errors, and in kubectl describe pod -n <<namespace>> <<podname>> the Containers/<<appname>> section includes

    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0

Which looks inconsistent - I can't see how it's got into CrashLoopBackoff with a last state of Terminated because Completed and an Exit Code of 0 - the application, as far as I can see isn't failing, it's just that Kubernetes is shutting down the pod as completed rather than leaving it running, and it's then somehow getting stuck in CrashLoopBackoff.

I'd wondered whether this was something to do with readiness or liveness probes killing it through not finding a perpetually running process to service a request, but removing them or reverting to older versions doesn't seem to make any difference.

There's presumably something that's gone wrong in the charts between the old and new versions, but I'm running out of ideas on where to look. Is there anything people can suggest that might be causing this?

moonkotte avatar
in flag
Is it any specific image that's used for this pod? What you described so far is absolutely correct. When process is ended, pod will be restarted by kubernetes. It depends on image/application inside the image. You can try both simple examples: 1st is busybox which will immediately ended and nginx which will continue running. Usually `sleep 86400` command is added to enforce pod from termination.
moonkotte avatar
in flag
Also you can share the manifest of this pod in x2 and x3 versions to see what's the difference (I didn't work with Jenkins, but know kubernetes itself as platform, so we can find something or workaround as above)
Wytrzymały Wiktor avatar
it flag
Hello @eftpotrm. Any updates?
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.