Does Kubernetes internal networking keep traffic to co-located Pods where possible?

In Kubernetes, if I run a Service Foo on every Node, and also Service Bar on every Node, i.e. with DaemonSets. If a Pod from Service Foo needs to make a request to Service Bar, does Kubernetes networking magic optimise things so communication will be done within the same Node, i.e. without a external network call? Or will a random Pod Bar be picked meaning usually Bar will be on a different Node, so there will be a network hop?

Context to question: I'm thinking to run CoreDNS as a Daemonset, because with the cache plugin, I'm hoping most DNS queries will not need to use the network as there will always be a CoreDNS Pod co-located within the same Node and most requests will be cached. Currently we only run 3 CoreDNS pods and we have 12 Nodes and we see some DNS failures at times of network traffic surges - particularly during a deployment when lot of logging is done, which have to be shipped across the network to ElasticSearch.

In case specifics matter:

• The CoreDNS service runs as clusterIP: 172.20.0.10

• The services that use CoreDNS run as type: NodePort

• The Kubernetes cluster is AWS EKS, v1.14.9. Networking is done with the AWS VPC CNI.

No, Kubernetes will load-balance requests to a ClusterIP across all Pods with matching labels (and passing readiness checks). It won't keep traffic within the same Node.

Q: So how does the NodeLocal DNS Cache achieve to keep DNS requests local to the Node?

A: When using the NodeLocal DNS Cache, it is necessary to pass a extra arg to the Kubelet of each Node (e.g. --kubelet-extra-args "--cluster-dns=169.254.20.10 ..."). This changes the resolv.conf of every container in every Pod to use DNS resolver on the Node, instead of a clusterIP. The NodeLocal Cache runs on every Node (as a DaemonSet) and uses a arg (e.g. "-localip", "169.254.20.10") to run on the specific IP set in the resolv.conf of all Pods. The NodeLocal DNS Cache Pod also needs to be configured with hostNetwork: true.

StackExchange DevOps Q#10776, answer score: 3