Skip to main content

Communication Errors

Background

  • ✅ MySQL can connect from its own pod.
  • ❌ MySQL can't connect by K8s service.
    • Service endpoint is right.
    • getent inside pod, sometimes failed
    • Pod network issue(Calico CNI).

Errors

JDBC Errors
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
...
The driver has not received any packets from the server
MySQL Errors
2025-07-29T02:45:56.595368Z 21528 [Note] Aborted connection 21528 to db: 'foo' user: 'root' host: '10.x.x.x' (Got an error reading communication packets)

Solutions

Check MySQL Inside

kubectl exec -it mysql-0 -- /bin/bash

mysql -uroot -p

Check Pod Network

If it sometimes fails, some CNI (Calico) pods might be down.

getent ahosts mysql-haproxy.ns.svc.cluster.local

10.x.x.x   STREAM mysql-haproxy.ns.svc.cluster.local
10.x.x.x   DGRAM
10.x.x.x   RAW

Check CNI(Calico) Pods

kubectl get pods | grep calico

calico-kube-controllers-6f697bd6bc-8s6df   1/1     Running   1 (17h ago)   27d
calico-node-jq9lc                          0/1     Running   0             6s
calico-node-ms4wt                          1/1     Running   2 (16h ago)   27d
calico-node-q4z9q                          1/1     Running   2 (16h ago)   27d

Delete calico-node-jq9lc to fix the problem.

kubectl delete pod calico-node-jq9lc

Check logs.

kubectl logs calico-node-ms4wt --previous

2025-07-28 09:20:42.273 [WARNING][96] felix/route_table.go 740: Failed to delete route error=no such process ifaceName="cali125e9a7ccf9" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2025-07-28 09:20:44.002 [INFO][96] felix/route_table.go 1205: Failed to access interface because it doesn't exist. error=Link not found ifaceName="cali6ac61bc7d83" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2025-07-28 09:20:44.002 [INFO][96] felix/route_table.go 1273: Failed to get interface; it's down/gone. error=Link not found ifaceName="cali6ac61bc7d83" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2025-07-28 09:21:08.572 [INFO][96] felix/route_table.go 1205: Failed to access interface because it doesn't exist. error=Link not found ifaceName="califb58b76f486" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2025-07-28 09:21:08.572 [INFO][96] felix/route_table.go 1273: Failed to get interface; it's down/gone. error=Link not found ifaceName="califb58b76f486" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2025-07-28 09:50:13.390 [INFO][98] confd/watchercache.go 125: Watch error received from Upstream ListRoot="/calico/ipam/v2/host/ingress-2" error=too old resource version: 1057 (7933983)
Back to top