Score:1

How to make Jenkins agents port available with an Ingress controller on Kubernetes?

cn flag

I'm trying to setup Jenkins in a Digital Ocean Kubernetes cluster. I'm using a NGINX ingress controller as I want to access my server from a subdomain (jenkins.example.com). Everything is working fine so far, I have my UI on said domain secured with a custom certificate. I started encountering problems when trying to connect my agents (or slaves I read elsewhere?) to the server.

What I've tried

I tried to setup a load balancer but couldn't make it work as I am using Digital Ocean and most of the docs I found were for GKE / EKS and others, which have their own internal LB. I also tried mapping a route /agents on the service's port name then on another service with a "ClusterIP" type, without success (not found from Jenkins).

My current config

jenkins-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jenkins-dep
  labels:
    app: jenkins
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jenkins
  template:
    metadata:
      labels:
        app: jenkins
    spec:
      containers:
      - name: jenkins
        image: jenkins/jenkins:lts-jdk11
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8080
        - containerPort: 50000

jenkins-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: jenkins-svc
  labels:
    app: jenkins
spec:
  type: NodePort
  selector:
    app: jenkins
  ports:
  - name: ui
    protocol: TCP
    port: 8080
    targetPort: 8080
    nodePort: 32500
  - name: agents
    protocol: TCP
    port: 50000
    targetPort: 50000
    nodePort: 32501

jenkins-ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: jenkins-ingress
spec:
  tls:
  - hosts:
      - jenkins.example.com
    secretName: tls-secret
  rules:
  - host: jenkins.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: jenkins-svc
            port:
              name: ui
      #- path: /agents
      #  pathType: Prefix
      #  backend:
      #    service:
      #      name: jenkins-svc
      #      port:
      #        name: agents

The bottom line is: how do I make available the Jenkins port 50000 for the agents to connect on my server? Or how could I change Jenkins settings to accommodate such a config?

Score:2
in flag

To the best of my knowledge, Jenkins uses a custom protocol for communicating with its build agents that is a binary Java RMI, and not HTTP based. Since the Ingress resource is only for host: based virtual dispatching of HTTP requests, you cannot use any kind: Ingress declaration for doing that

But, with the nginx-ingress controller specifically, they do allow exposing TCP services which will likely do what you want

greg_wss avatar
cn flag
Thank you for this. I'm currently following their doc, I'll update this post if I manage to achieve anything.
andolsi zied avatar
jp flag
Effectively, the solution is to expose the TCP service. To avoid errors, I recommend using the helm release from ingress-nginx (I tested 4.0.12) like described here (https://docs.cloudbees.com/docs/cloudbees-ci/latest/cloud-setup-guide/configure-ports-jnlp-agents#values-yaml-configure-jnlp)
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.