Score:0

Server

Pod running kubectl trying to reach external cluster

brgsousa

7/22/23, 3:24 PM

I am using a pod (image bitnami/kubectl) in a kubernetes cluster (clusterA) but kubeconfig is set to point to another cluster (clusterB).

Using on premises kubernetes 1.21.7 (on VMs) installed via kubeadm.

kubeconfig used:

apiVersion: v1
kind: Config
clusters:
- name: default-cluster
  cluster:
    certificate-authority-data: XXXXXXXXXXXXXXXXXXXXXX
    server: https://clusterB:6443
contexts:
- name: default-context
  context:
    cluster: default-cluster
    namespace: default
    user: default-user
current-context: default-context
users:
- name: default-user
  user:
    token: YYYYYYYYYYYYYYYYYYYYYY

But even though kubeconfig is correct, clusterA doesn't allow the request to leave clusterA to reach clusterB. It seems kubernetes control plane in clusterA interprets the pod kubectl request as if it is trying to control clusterA itself but that's not what I am trying to do. I am trying to reach clusterB instead (defined in 'server' in kubeconfig).

This is the error when kubectl runs:

Error from server (Forbidden): error when retrieving current configuration of:
Resource: "/v1, Resource=secrets", GroupVersionKind: "/v1, Kind=Secret"
Name: "mysecret", Namespace: "istio-system"
from server for: "STDIN": secrets "mysecret" is forbidden: User "system:serviceaccount:mynamespace:default" cannot get resource "secrets" in API group "" in the namespace "istio-system"

EDIT: I am using kubectl setting a custom kubeconfig:

# kubectl --kubeconfig=/etc/custom.kubeconfig cluster-info
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
Error from server (Forbidden): services is forbidden: User "system:serviceaccount:mynamespace:default" cannot list resource "services" in API group "" in the namespace "kube-system"

# grep server /etc/letsencrypt/custom.kubeconfig
server: https://clusterB:6443

EDIT2:

# kubectl --kubeconfig=/etc/custom.kubeconfig cluster-info dump
Error from server (Forbidden): nodes is forbidden: User "system:serviceaccount:cadeado--producao:default" cannot list resource "nodes" in API group "" at the cluster scope

0 + 0

cluster

kubernetes

kubectl

Mikołaj Głodziak

7/23/23, 12:21 PM

Which version of Kubernetes did you use and how did you set up the cluster? Did you use bare metal installation or some cloud provider? It is important to reproduce your problem.

brgsousa

7/23/23, 12:49 PM

Using on premises kubernetes 1.21.7 (on VMs) installed via kubeadm.

Mikołaj Głodziak

7/25/23, 1:05 PM

Did you try to run this command: `kubectl cluster-info dump`? Could you paste the results to the question?

brgsousa

7/30/23, 7:45 PM

I added a second edit

Mikołaj Głodziak

7/31/23, 7:44 AM

Please see [this similar problem](https://stackoverflow.com/questions/69535118/user-systemserviceaccountdefaultflink-cannot-list-resource-nodes-in-api-g). It is related to another product, but in the answer you will get information what should you check and how can you repair your problem. Please let me know if it was helpful.

brgsousa

7/31/23, 6:50 PM

Kind of. The issue in that post is inside the same cluster. What I am trying to do here is a "cross-cluster" access. Pod in cluster A is trying to access cluster B

Mikołaj Głodziak

8/1/23, 9:42 AM

I have one more [similar problem](https://stackoverflow.com/questions/67151953/forbidden-resource-in-api-group-at-the-cluster-scope) to yours. If it will be not helpful please run the command `kubectl auth can-i list nodes <insert here your user>` (similar to the command from linked question) and paste here results.

Score:0

Server

gayan ranasinghe

7/23/23, 9:21 AM

After issuing following from pod:

kubectl cluster-info

what do you get?

Seems the problem you have is related to user access, you need to provide
default-user with appropriate RBAC rights.

0 + 0

brgsousa

7/23/23, 12:50 PM

Got this error:To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'. Error from server (Forbidden): services is forbidden: User "system:serviceaccount:mynamespace:default" cannot list resource "services" in API group "" in the namespace "kube-system"

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Pod running kubectl trying to reach external cluster

TH: พ็อดที่รัน kubectl พยายามเข้าถึงคลัสเตอร์ภายนอก

RO: Pod rulează kubectl încercând să ajungă la un cluster extern

RU: Под, работающий с kubectl, пытается получить доступ к внешнему кластеру

VI: Pod đang chạy kubectl đang cố truy cập cụm bên ngoài

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.