Rancher PVC Pending

因为 local path storage 在动态创建 helper pod 的时候，没有成功拉到 helper pod 的镜像

Errors

PVC Pending

code-gitaly-data-foobar-code-gitaly-0-2-0  Pending local-path  216d

Describe pvc.

Type:        Normal
Reason:      ExternalProvisioning
Age:         4m32s (x15836 over 2d18h)
From:        persistentvolume-controller
Message:     waiting for a volume to be created, either by external provisioner "rancher.io/local-path" or manually created by system administrator

Pod logs: local-path-provisioner

I0114 09:56:11.417552       1 controller.go:1202] provision "monitor/data-foobar-kafka-controller-0" class "local-path": started
time="2025-01-14T09:56:11Z" level=debug msg="config doesn't contain node instance-smuk9oo0-2, use DEFAULT_PATH_FOR_NON_LISTED_NODES instead"
time="2025-01-14T09:56:11Z" level=info msg="Creating volume pvc-2e00c1b0-57fd-4fd9-b6a0-7e46e9427af2 at instance-smuk9oo0-2:/opt/local-path-provisioner/pvc-2e00c1b0-57fd-4fd9-b6a0-7e46e9427af2_monitor_data-foobar-kafka-controller-0"
time="2025-01-14T09:56:11Z" level=info msg="create the helper pod helper-pod-create-pvc-2e00c1b0-57fd-4fd9-b6a0-7e46e9427af2 into local-path-storage"
I0114 09:56:11.421637       1 event.go:281] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"monitor", Name:"data-foobar-kafka-controller-0", UID:"2e00c1b0-57fd-4fd9-b6a0-7e46e9427af2", APIVersion:"v1", ResourceVersion:"561315142", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "monitor/data-foobar-kafka-controller-0"
W0114 09:58:12.298943       1 controller.go:893] Retrying syncing claim "2e00c1b0-57fd-4fd9-b6a0-7e46e9427af2" because failures 3 < threshold 15
E0114 09:58:12.298972       1 controller.go:913] error syncing claim "2e00c1b0-57fd-4fd9-b6a0-7e46e9427af2": failed to provision volume with StorageClass "local-path": failed to create volume pvc-2e00c1b0-57fd-4fd9-b6a0-7e46e9427af2: create process timeout after 120 seconds
I0114 09:58:12.299016       1 event.go:281] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"monitor", Name:"data-foobar-kafka-controller-0", UID:"2e00c1b0-57fd-4fd9-b6a0-7e46e9427af2", APIVersion:"v1", ResourceVersion:"561315142", FieldPath:""}): type: 'Warning' reason: 'ProvisioningFailed' failed to provision volume with StorageClass "local-path": failed to create volume pvc-2e00c1b0-57fd-4fd9-b6a0-7e46e9427af2: create process timeout after 120 seconds

Describe rancher helper pod.

Failed to pull image "docker-hub-nexus.foobar.work/foobar-bravo-x86/busybox:latest": rpc error: code = Unknown desc = Error response from daemon: Get https://docker-hub-nexus.foobar.work/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Error: ErrImagePull
Back-off pulling image "docker-hub-nexus.foobar.work/foobar-bravo-x86/busybox:latest"
Error: ImagePullBackOff
Pulling image "docker-hub-nexus.foobar.work/foobar-bravo-x86/busybox:latest"