I've spent my last 2 days trying to figure out what's wrong with my setup and couldn't find a resonable answer or solution, so I want to share it with you.
What happened: I've deployed a new etcd using helm and configured external-dns to create dns records inside it, and also set-up a coredns deployment to read records from etcd.
but, the problem is, when the records are created inside etcd, coredns wouldn't response to my dig
commands, in short, i think coredns doesn't recognize queries with unempty "text" keys.
external-dns creates these items:
/opt/bitnami/etcd$ etcdctl get --prefix '/skydns'
/skydns/com/aminm/cname-goh/3ee06812
{"text":"\"heritage=external-dns,external-dns/owner=staging,external-dns/resource=ingress/stg-infra/goh.aminm.com-k8s\"","targetstrip":1}
/skydns/com/aminm/goh/149a1668
{"host":"sag.com","text":"\"heritage=external-dns,external-dns/owner=staging,external-dns/resource=ingress/stg-infra/goh.aminm.com-k8s\"","targetstrip":1}
and dnstools returns no result:
dnstools# dig @10.103.49.243 goh.aminm.com
; <<>> DiG 9.11.3 <<>> @10.103.49.243 goh.aminm.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 52879
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: f75cd784c97c1fd1 (echoed)
;; QUESTION SECTION:
;goh.aminm.com. IN A
;; AUTHORITY SECTION:
aminm.com. 30 IN SOA ns.dns.aminm.com. hostmaster.aminm.com. 1684880221 7200 1800 86400 30
;; Query time: 9 msec
;; SERVER: 10.103.49.243#53(10.103.49.243)
;; WHEN: Tue May 23 22:17:01 UTC 2023
;; MSG SIZE rcvd: 135
coredns log:
[INFO] 10.35.0.21:48154 - 29227 "A IN goh.aminm.com. udp 54 false 4096" NOERROR qr,aa,rd 112 0.002888737s
but when i manually create a new record and specify no "text" value in etcd:
/opt/bitnami/etcd$ etcdctl put /skydns/com/aminm/goh/3031e6b8 '{"host":"sag.com","text":"","targetstrip":1}'
dig command works correctly:
dnstools# dig @10.103.49.243 goh.aminm.com
; <<>> DiG 9.11.3 <<>> @10.103.49.243 goh.aminm.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 47171
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;goh.aminm.com. IN A
;; ANSWER SECTION:
goh.aminm.com. 1296000 IN A 66.94.118.137
;; Query time: 242 msec
;; SERVER: 10.103.49.243#53(10.103.49.243)
;; WHEN: Tue May 23 22:23:40 UTC 2023
;; MSG SIZE rcvd: 71
also this is my coredns configuration:
.:53 {
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
fallthrough
ttl 30
}
etcd aminm.com {
path /skydns
endpoint http://dns-etcd.stg-infra.svc.cluster.local:2379
fallthrough
}
errors
health {
lameduck 5s
}
ready
log
prometheus 0.0.0.0:9153
forward . 1.1.1.1
reload
}
I expect coredns to be able to read and response to etcd keys even if they have "test" key inside them