Skip to main content

Communication Errors

Background

  • ✅ MySQL can connect from its own pod.
  • ❌ MySQL can't connect by K8s service.
    • Service endpoint is right.
    • getent inside pod, sometimes failed
    • Pod network issue(Calico CNI).

Errors

JDBC Errors
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
...
The driver has not received any packets from the server
MySQL Errors
2025-07-29T02:45:56.595368Z 21528 [Note] Aborted connection 21528 to db: 'foo' user: 'root' host: '10.x.x.x' (Got an error reading communication packets)

Solutions

Check MySQL Inside

kubectl exec -it mysql-0 -- /bin/bash

mysql -uroot -p

Check Pod Network

If it sometimes fails, some CNI (Calico) pods might be down.

getent ahosts mysql-haproxy.ns.svc.cluster.local

10.x.x.x   STREAM mysql-haproxy.ns.svc.cluster.local
10.x.x.x   DGRAM
10.x.x.x   RAW

Check CNI(Calico) Pods

kubectl get pods | grep calico

calico-kube-controllers-6f697bd6bc-8s6df   1/1     Running   1 (17h ago)   27d
calico-node-jq9lc                          0/1     Running   0             6s
calico-node-ms4wt                          1/1     Running   2 (16h ago)   27d
calico-node-q4z9q                          1/1     Running   2 (16h ago)   27d

Delete calico-node-jq9lc to fix the problem.

kubectl delete pod calico-node-jq9lc

Calico Error Logs

kubectl logs calico-node-ms4wt --previous

2025-07-28 09:20:42.273 [WARNING][96] felix/route_table.go 740: Failed to delete route error=no such process ifaceName="cali125e9a7ccf9" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2025-07-28 09:20:44.002 [INFO][96] felix/route_table.go 1205: Failed to access interface because it doesn't exist. error=Link not found ifaceName="cali6ac61bc7d83" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2025-07-28 09:20:44.002 [INFO][96] felix/route_table.go 1273: Failed to get interface; it's down/gone. error=Link not found ifaceName="cali6ac61bc7d83" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2025-07-28 09:21:08.572 [INFO][96] felix/route_table.go 1205: Failed to access interface because it doesn't exist. error=Link not found ifaceName="califb58b76f486" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2025-07-28 09:21:08.572 [INFO][96] felix/route_table.go 1273: Failed to get interface; it's down/gone. error=Link not found ifaceName="califb58b76f486" ifaceRegex="^cali.*" ipVersion=0x4 tableIndex=0
2025-07-28 09:50:13.390 [INFO][98] confd/watchercache.go 125: Watch error received from Upstream ListRoot="/calico/ipam/v2/host/ingress-2" error=too old resource version: 1057 (7933983)

Failed to Get Interface

Failed to get interface; it's down/gone. error=Link not found ifaceName="cali125e9a7ccf9"

Calico 想处理某个接口（如 cali125e9a7ccf9），但内核返回接口已不存在。

  • 相关 Pod 被删除，但 Calico 还没完全清理对应路由。
  • 或 Pod 快速重建时，旧接口消失，Felix 还没感知。

Failed to Delete Route

Failed to delete route error=no such process ifaceName="cali125e9a7ccf9"

Calico 想删掉某个与该接口绑定的路由，但系统报错 no such process。 这是一种无害的警告，意味着系统状态已提前变更，Calico 的清理动作重复了一次。

Too Old Resource Version

Watch error received from Upstream ListRoot="/calico/ipam/v2/host/ingress-2" error=too old resource version

confd（Calico 内部的配置同步模块）从 etcd（或 kube-apiserver）watch 数据失败，提示 “resourceVersion 太旧”。 这是 Kubernetes watch 机制的正常现象，如果版本太老（如本地断网或负载高导致滞后），会断掉重新全量拉取。

Back to top