时间:2023-02-06 11:32:01 | 来源:建站知识
时间:2023-02-06 11:32:01 来源:建站知识
微信公众号:运维开发故事,作者:姜总
root@demo-hello-perf-dev-v0-5-0-f9f9cd5c9-r27cw:/# cat /etc/resolv.confnameserver 10.10.0.2search paas.svc.cluster.local svc.cluster.local cluster.localoptions ndots:5
在该容器中安装 nslookup 工具,然后对 http://www.ayunw.cn 域名进行解析[root@kube-master-srv1 ~]# kubectl get po -n paasNAME READY STATUS RESTARTS AGEdemo-hello-perf-dev-v0-5-0-f9f9cd5c9-r27cw 1/1 Running 0 11d[root@kube-master-srv1 ~]# kubectl exec -it demo-hello-perf-dev-v0-5-0-f9f9cd5c9-r27cw -n paas -- bashroot@demo-hello-perf-dev-v0-5-0-f9f9cd5c9-r27cw:/# cat /etc/issueDebian GNU/Linux 10 /n /lroot@demo-hello-perf-dev-v0-5-0-f9f9cd5c9-r27cw:/# apt -y install dnsutils
接着找到某一个coredns,然后去他所调度到的node节点通过nsenter进入网络名称空间进行抓包分析# 在k8s-master上查看coredns调度在哪个node# 接着我就选择了第一个coredns[root@kube-master-srv1 ~]# kubectl get po -n kube-system -o wide | grep corednscoredns-69d9b6c494-4nrxt 1/1 Running 0 96d 10.20.246.18 node2.core <none> <none>coredns-69d9b6c494-6vjw4 1/1 Running 0 96d 10.20.240.239 node3.core <none> <none>coredns-69d9b6c494-pw5gx 1/1 Running 0 96d 10.20.240.232 node3.core <none> <none># 登录到 node2.core 节点,找到coredns的pid# 进入这个pid进入coredns容器的网络名称空间进行抓包过滤分析[root@kube-node-srv2 ~]# docker ps -a | grep coredns4d38fd311a78 bfe3a36ebd25 "/coredns -conf /etc…" 3 months ago Up 3 months k8s_coredns_coredns-69d9b6c494-4nrxt_kube-system_803290a5-b4bd-4f2e-81b3-5ce82c9aa57c_000722e50786b registry.xx.xx/library/k8s.gcr.io/pause:3.2 "/pause" 3 months ago Up 3 months k8s_POD_coredns-69d9b6c494-4nrxt_kube-system_803290a5-b4bd-4f2e-81b3-5ce82c9aa57c_0[root@kube-node-srv2 ~]# docker inspect -f {{.State.Pid}} 4d38fd311a78896949[root@kube-node-srv2 ~]# nsenter -n -t 896949[root@kube-node-srv2 ~]# ifconfigeth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1380 inet 10.20.246.18 netmask 255.255.255.255 broadcast 10.20.246.18 ether 46:c1:e0:30:b4:9d txqueuelen 0 (Ethernet) RX packets 1489941923 bytes 162419228606 (151.2 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1488233127 bytes 297011464372 (276.6 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 loop txqueuelen 1000 (Local Loopback) RX packets 83731165 bytes 6681735331 (6.2 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 83731165 bytes 6681735331 (6.2 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@kube-master-srv1 ~]# kubectl get svc kubernetesNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEkubernetes ClusterIP 10.10.0.1 <none> 443/TCP 57droot@demo-hello-perf-dev-v0-5-0-f9f9cd5c9-r27cw:/# nslookup kubernetes.defaultServer: 10.10.0.2Address: 10.10.0.2#53Name: kubernetes.default.svc.cluster.localAddress: 10.10.0.1
[root@kube-node-srv2 ~]# tcpdump -i eth0 port 53 | grep "kubernetes"tcpdump: verbose output suppressed, use -v or -vv for full protocol decodelistening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes16:44:42.712421 IP 10.20.105.252.60020 > qing-core-kube-node-srv2.domain: 7282+ A? kubernetes.default.svc.cluster.local. (54)16:44:48.883881 IP 10.20.105.252.ndm-agent-port > qing-core-kube-node-srv2.domain: 25500+ AAAA? kubernetes.default.svc.cluster.local. (54)16:50:15.361021 IP 10.20.105.252.57205 > qing-core-kube-node-srv2.domain: 24061+ A? kubernetes.default.paas.svc.cluster.local. (59)16:50:22.186723 IP 10.20.105.252.60715 > qing-core-kube-node-srv2.domain: 55799+ AAAA? kubernetes.default.svc.cluster.local. (54)16:50:27.813477 IP qing-core-kube-node-srv2.domain > 10.20.176.128.8181: 21787*- 1/0/0 PTR kubernetes.default.svc.cluster.local. (112)16:46:04.429250 IP 10.20.105.252.33895 > qing-core-kube-node-srv2.domain: 37943+ A? kubernetes.default.svc.cluster.local.svc.cluster.local. (72)16:46:04.441717 IP 10.20.105.252.54502 > qing-core-kube-node-srv2.domain: 45454+ AAAA? kubernetes.default.svc.cluster.local. (54)16:46:10.771445 IP 10.20.105.252.54594 > qing-core-kube-node-srv2.domain: 16257+ A? kubernetes.default.svc.cluster.local.svc.cluster.local. (72)16:46:10.783322 IP 10.20.105.252.59768 > qing-core-kube-node-srv2.domain: 60408+ AAAA? kubernetes.default.svc.cluster.local. (54)
通过以上抓包分析得出结论。当解析kubernetes域名的时候,点的个数比ndots的值小,则按照search后面的本地域参数填补了域名后缀,当按照顺序 用 paas.svc.cluster.local 填补的时候解析到了A记录。然后终止dns查询将查询到的A记录返回。root@demo-hello-pro-master-5474b97bdf-fvbm5:/# host -v kubernetes.defaultTrying "kubernetes.default.paas.svc.cluster.local"Trying "kubernetes.default.svc.cluster.local";; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18054;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0;; QUESTION SECTION:;kubernetes.default.svc.cluster.local. IN A;; ANSWER SECTION:kubernetes.default.svc.cluster.local. 5 IN A 10.10.0.1Received 106 bytes from 10.10.0.2#53 in 3 msTrying "kubernetes.default.svc.cluster.local";; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58952;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0;; QUESTION SECTION:;kubernetes.default.svc.cluster.local. IN AAAA;; AUTHORITY SECTION:cluster.local. 5 IN SOA ns.dns.cluster.local. hostmaster.cluster.local. 1622445553 7200 1800 86400 5Received 147 bytes from 10.10.0.2#53 in 2 msTrying "kubernetes.default.svc.cluster.local";; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37783;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0;; QUESTION SECTION:;kubernetes.default.svc.cluster.local. IN MX;; AUTHORITY SECTION:cluster.local. 5 IN SOA ns.dns.cluster.local. hostmaster.cluster.local. 1622445553 7200 1800 86400 5Received 147 bytes from 10.10.0.2#53 in 2 ms
root@demo-hello-perf-dev-v0-5-0-f9f9cd5c9-r27cw:/# nslookup www.ayunw.cnServer: 10.10.0.2Address: 10.10.0.2#53Non-authoritative answer:Name: www.ayunw.cnAddress: 134.175.123.64
[root@kube-node-srv2 ~]# tcpdump -i eth0 port 53 | grep "ayunw"tcpdump: verbose output suppressed, use -v or -vv for full protocol decodelistening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes14:38:07.350640 IP 10.20.105.252.47767 > qing-core-kube-node-srv2.domain: 13102+ A? www.ayunw.cn.cluster.local. (44)14:38:19.098753 IP 10.20.105.252.47071 > qing-core-kube-node-srv2.domain: 15535+ A? www.ayunw.cn.paas.svc.cluster.local. (53)14:38:19.111441 IP 10.20.105.252.56968 > qing-core-kube-node-srv2.domain: 62838+ A? www.ayunw.cn. (30)14:38:19.111720 IP qing-core-kube-node-srv2.35187 > 172.16.0.11.domain: 62838+ A? www.ayunw.cn. (30)14:38:31.200982 IP 10.20.105.252.50777 > qing-core-kube-node-srv2.domain: 10715+ A? www.ayunw.cn.svc.cluster.local. (48)14:38:31.214096 IP 10.20.105.252.51233 > qing-core-kube-node-srv2.domain: 37585+ AAAA? www.ayunw.cn. (30)14:38:31.214299 IP qing-core-kube-node-srv2.35187 > 172.16.0.11.domain: 37585+ AAAA? www.ayunw.cn. (30)14:39:04.691754 IP 10.20.105.252.34080 > qing-core-kube-node-srv2.domain: 34206+ A? www.ayunw.cn.paas.svc.cluster.local. (53)14:39:04.704758 IP 10.20.105.252.36478 > qing-core-kube-node-srv2.domain: 64751+ A? www.ayunw.cn. (30)14:39:04.705068 IP qing-core-kube-node-srv2.48926 > 172.16.0.11.domain: 64751+ A? www.ayunw.cn. (30)14:39:13.925872 IP 10.20.105.252.59868 > qing-core-kube-node-srv2.domain: 45121+ A? www.ayunw.cn.paas.svc.cluster.local. (53)14:39:13.937328 IP 10.20.105.252.45290 > qing-core-kube-node-srv2.domain: 27511+ A? www.ayunw.cn. (30)14:39:13.937576 IP qing-core-kube-node-srv2.48926 > 172.16.0.11.domain: 27511+ A? www.ayunw.cn. (30)14:39:24.838444 IP 10.20.105.252.37510 > qing-core-kube-node-srv2.domain: 45926+ A? www.ayunw.cn.cluster.local. (44)14:45:13.438961 IP 10.20.105.252.55462 > qing-core-kube-node-srv2.domain: 60170+ A? www.ayunw.cn.paas.svc.cluster.local. (53)14:45:13.450865 IP 10.20.105.252.42674 > qing-core-kube-node-srv2.domain: 25680+ A? www.ayunw.cn. (30)14:45:13.451110 IP qing-core-kube-node-srv2.56396 > 172.16.0.11.domain: 25680+ A? www.ayunw.cn. (30)^C35952 packets captured35956 packets received by filter0 packets dropped by kernel
从上面抓包分析的结果来看, http://www.ayunw.cn 的这个域名只有两个点,比pod里面 /etc/resolv.conf 文件中的 ndots 配置的值小(ndots的值为5,域名的点为2)。则会按照search的参数填补域名后缀,并且是根据search后面的顺序 paas.svc.cluster.local 、 svc.cluster.local 、 cluster.local 依次来填充的。因为根据search后面的本地域匹配后都没有域名解析的结果,因此他就直接解析了 http://www.ayunw.cn 这个域名查询到了该域名的A记录并且返回了结果。root@demo-hello-pro-master-5474b97bdf-fvbm5:/# host -v www.ayunw.cnTrying "www.ayunw.cn.paas.svc.cluster.local"Trying "www.ayunw.cn.svc.cluster.local"Trying "www.ayunw.cn.cluster.local"Trying "www.ayunw.cn";; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8135;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 13, ADDITIONAL: 0;; QUESTION SECTION:;www.ayunw.cn. IN A;; ANSWER SECTION:www.ayunw.cn. 30 IN A 134.175.123.64;; AUTHORITY SECTION:. 30 IN NS l.root-servers.net.. 30 IN NS e.root-servers.net.. 30 IN NS h.root-servers.net.. 30 IN NS k.root-servers.net.. 30 IN NS d.root-servers.net.. 30 IN NS b.root-servers.net.. 30 IN NS g.root-servers.net.. 30 IN NS j.root-servers.net.. 30 IN NS m.root-servers.net.. 30 IN NS i.root-servers.net.. 30 IN NS f.root-servers.net.. 30 IN NS c.root-servers.net.. 30 IN NS a.root-servers.net.Received 461 bytes from 10.10.0.2#53 in 94 msTrying "www.ayunw.cn";; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11085;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0;; QUESTION SECTION:;www.ayunw.cn. IN AAAA;; AUTHORITY SECTION:ayunw.cn. 5 IN SOA dns17.hichina.com. hostmaster.hichina.com. 2019070911 3600 1200 86400 360Received 113 bytes from 10.10.0.2#53 in 99 msTrying "www.ayunw.cn";; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19432;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0;; QUESTION SECTION:;www.ayunw.cn. IN MX;; AUTHORITY SECTION:ayunw.cn. 5 IN SOA dns17.hichina.com. hostmaster.hichina.com. 2019070911 3600 1200 86400 360Received 113 bytes from 10.10.0.2#53 in 51 ms
因为我的pod中存在三个本地域:paas.svc.cluster.local 、 svc.cluster.local 、 cluster.local ,通过host命令可以看到,Trying 一共尝试了四次,一次根据我search后面的本地域进行了解析搜索,结果没有搜索到正确的解析,因此通过pod所在的宿主机本地的 /etc/resolv.conf 文件中进行了解析。# cat /etc/resolv.confoptions rotate timeout:1; generated by /usr/sbin/dhclient-scriptnameserver 172.16.0.11nameserver 172.16.0.12
解析 http://www.jd.com 域名root@demo-hello-perf-dev-v0-5-0-f9f9cd5c9-r27cw:/# nslookup www.jd.comServer: 10.10.0.2Address: 10.10.0.2#53Non-authoritative answer:www.jd.com canonical name = www.jd.com.gslb.qianxun.com.www.jd.com.gslb.qianxun.com canonical name = www.jdcdn.com.www.jdcdn.com canonical name = img20.360buyimg.com.s.galileo.jcloud-cdn.com.img20.360buyimg.com.s.galileo.jcloud-cdn.com canonical name = img2x-sched.jcloud-cdn.com.Name: img2x-sched.jcloud-cdn.comAddress: 113.107.249.3
以下是抓http://www.jd.com的域名DNS包的结果:[root@kube-node-srv2 ~]# tcpdump -i eth0 port 53 | grep "jd"tcpdump: verbose output suppressed, use -v or -vv for full protocol decodelistening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes16:17:52.935226 IP 10.20.105.252.56775 > qing-core-kube-node-srv2.domain: 17278+ A? www.jd.com.paas.svc.cluster.local. (51)16:17:52.947890 IP 10.20.105.252.52012 > qing-core-kube-node-srv2.domain: 12806+ A? www.jd.com. (28)16:17:52.948150 IP qing-core-kube-node-srv2.54626 > 172.16.0.11.domain: 12806+ A? www.jd.com. (28)16:17:53.054427 IP 172.16.0.11.domain > qing-core-kube-node-srv2.54626: 12806 5/13/0 CNAME www.jd.com.gslb.qianxun.com., CNAME www.jdcdn.com., CNAME img20.360buyimg.com.s.galileo.jcloud-cdn.com., CNAME img2x-sched.jcloud-cdn.com., A 113.107.249.3 (398)16:17:53.054677 IP qing-core-kube-node-srv2.domain > 10.20.105.252.52012: 12806 5/13/0 CNAME www.jd.com.gslb.qianxun.com., CNAME www.jdcdn.com., CNAME img20.360buyimg.com.s.galileo.jcloud-cdn.com., CNAME img2x-sched.jcloud-cdn.com., A 113.107.249.3 (398)
通过host命令检测http://www.jd.com和上面http://www.ayunw.cn一样的root@demo-hello-pro-master-5474b97bdf-fvbm5:/# host -v www.jd.comTrying "www.jd.com.paas.svc.cluster.local"Trying "www.jd.com.svc.cluster.local"Trying "www.jd.com.cluster.local"Trying "www.jd.com";; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 61910;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 13, ADDITIONAL: 0;; QUESTION SECTION:;www.jd.com. IN A;; ANSWER SECTION:www.jd.com. 13 IN CNAME www.jd.com.gslb.qianxun.com.www.jd.com.gslb.qianxun.com. 13 IN CNAME www.jdcdn.com.www.jdcdn.com. 13 IN CNAME img20.360buyimg.com.s.galileo.jcloud-cdn.com.img20.360buyimg.com.s.galileo.jcloud-cdn.com. 13 IN CNAME img2x-sched.jcloud-cdn.com.img2x-sched.jcloud-cdn.com. 13 IN A 113.107.249.3;; AUTHORITY SECTION:. 13 IN NS f.root-servers.net.. 13 IN NS i.root-servers.net.. 13 IN NS d.root-servers.net.. 13 IN NS l.root-servers.net.. 13 IN NS j.root-servers.net.. 13 IN NS g.root-servers.net.. 13 IN NS k.root-servers.net.. 13 IN NS m.root-servers.net.. 13 IN NS h.root-servers.net.. 13 IN NS c.root-servers.net.. 13 IN NS a.root-servers.net.. 13 IN NS e.root-servers.net.. 13 IN NS b.root-servers.net.Received 398 bytes from 10.10.0.2#53 in 5 msTrying "img2x-sched.jcloud-cdn.com";; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64422;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0;; QUESTION SECTION:;img2x-sched.jcloud-cdn.com. IN AAAA;; AUTHORITY SECTION:jcloud-cdn.com. 5 IN SOA ns1.jdgslb.com. apollo.jdgslb.com. 1622435242 10800 3600 604800 3600Received 125 bytes from 10.10.0.2#53 in 4 msTrying "img2x-sched.jcloud-cdn.com";; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43091;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0;; QUESTION SECTION:;img2x-sched.jcloud-cdn.com. IN MX;; AUTHORITY SECTION:jcloud-cdn.com. 5 IN SOA ns1.jdgslb.com. apollo.jdgslb.com. 1622435242 10800 3600 604800 3600Received 125 bytes from 10.10.0.2#53 in 40 ms
root@demo-hello-perf-dev-v0-5-0-f9f9cd5c9-r27cw:/# nslookup x.y.z.v.ayunw.cnServer: 10.10.0.2Address: 10.10.0.2#53Non-authoritative answer:Name: x.y.z.v.ayunw.cnAddress: 134.175.123.64
[root@kube-node-srv2 ~]# tcpdump -i eth0 port 53 | grep "ayunw"tcpdump: verbose output suppressed, use -v or -vv for full protocol decodelistening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes16:36:49.928116 IP 10.20.105.252.46581 > qing-core-kube-node-srv2.domain: 38769+ A? x.y.z.v.ayunw.cn. (34)16:36:49.928383 IP qing-core-kube-node-srv2.59801 > 172.16.0.11.domain: 38769+ A? x.y.z.v.ayunw.cn. (34)16:36:56.901762 IP 10.20.105.252.43844 > qing-core-kube-node-srv2.domain: 3524+ A? x.y.z.v.ayunw.cn. (34)16:37:01.763743 IP 10.20.105.252.36053 > qing-core-kube-node-srv2.domain: 62952+ AAAA? x.y.z.v.ayunw.cn. (34)16:37:01.764110 IP qing-core-kube-node-srv2.59801 > 172.16.0.11.domain: 62952+ AAAA? x.y.z.v.ayunw.cn. (34)16:37:06.851820 IP 10.20.105.252.36305 > qing-core-kube-node-srv2.domain: 58393+ AAAA? x.y.z.v.ayunw.cn. (34)16:37:06.852118 IP qing-core-kube-node-srv2.59801 > 172.16.0.11.domain: 58393+ AAAA? x.y.z.v.ayunw.cn. (34)
从上述抓包结果可以看到,如果域名中的点等于ndots的值,他会直接解析域名,不会用search后面的本地域来填补的。可能因为我阿里云上这个域名的原因,不支持超过5个点的域名解析。所以超过5个点的域名我无法测试。关键词: