Score:0

Enable Vault JWT using `-tls-skip-verify` with EKS ca.crt fails with `x509: certificate signed by unknown authority`

za flag
Jim

We need to enable JWT auth in vault which is hosted within our EKS cluster in preparation for using K8s 1.24 OIDC and testing token renewal with Vault. I'm following documentation from a few places:

https://banzaicloud.com/blog/kubernetes-oidc/

https://www.vaultproject.io/docs/auth/jwt/oidc-providers/kubernetes

https://www.vaultproject.io/docs/auth/kubernetes#discovering-the-service-account-issuer

https://learn.hashicorp.com/tutorials/vault/agent-kubernetes?in=vault/auth-methods

The configuration seems pretty straightforward. Where I'm lost is when it's time to write the JWT configuration to vault, the configuration fails with:

vault write -tls-skip-verify auth/jwt/config oidc_discovery_url=https://oidc.eks.us-west-2.amazonaws.com/id/xxxxxxxxxxxxxxxxxxxxxx oidc_discovery_ca_pem=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt bound_issuer=https://oidc.eks.us-west-2.amazonaws.com/id/xxxxxxxxxxxxxxxxxxxxxx
...
...
...
* error checking oidc discovery URL: error creating provider with given values: Get "https://vault.vault:8200/.well-known/openid-configuration": x509: certificate signed by unknown authority

Grabbing the CA:

at 22:37:06 ❯ \kubectl config view --raw --minify --flatten     --output 'jsonpath={.clusters[].cluster.certificate-authority-data}' | base64 --decode | openssl x509 -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 0 (0x0)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = kubernetes
        Validity
            Not Before: Oct 15 05:07:03 2022 GMT
            Not After : Oct 12 05:07:03 2032 GMT
        Subject: CN = kubernetes
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)

The CA used there is the CA which is provided everywhere for the cluster. It's in the kubeconfig, in the issuer for the cluster, and mounted with the serviceaccount data within pods as is what I used in the above command. There are no other CAs of which I'm aware. Moreover, using -tls-skip-verify is not honored, so that confuses me a little.

Any pointers on what's going on here would be greatly appreciated.

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.