Skip to content

Capturing the Kubernetes event on failing to pull plugin images #136

@gemblerz

Description

@gemblerz

The scheduler does not seem to capture the following event well. As a result, the portal shows the job is still running. The scheduler needs to handle this error.

DEBUG: 2026/04/07 22:49:16 nodescheduler.go:542: &Event{ObjectMeta:{panda-lidar-sched-5563.18a4316ce628d157  ses  608e00c1-9167-4cbf-8809-3ae21fa0e89d 53277591 0 2026-04-07 21:48:16 +0000 UTC <nil> <nil> map[] map[] [] []  [{k3s Update v1 2026-04-07 21:48:16 +0000 UTC FieldsV1 {"f:count":{},"f:firstTimestamp":{},"f:involvedObject":{},"f:lastTimestamp":{},"f:message":{},"f:reason":{},"f:source":{"f:component":{},"f:host":{}},"f:type":{}} }]},InvolvedObject:ObjectReference{Kind:Pod,Namespace:ses,Name:panda-lidar-sched-5563,UID:ee593047-32bb-4e9d-9c5e-76f097ee27fc,APIVersion:v1,ResourceVersion:53266205,FieldPath:spec.containers{panda-lidar-sched},},Reason:BackOff,Message:Back-off pulling image "registry.gitlab.com/lbl-anp/panda/dawn_pipeline/plugin-lidar:1.0.4",Source:EventSource{Component:kubelet,Host:000048b02d3af487.ws-nxcore,},FirstTimestamp:2026-04-07 21:48:16 +0000 UTC,LastTimestamp:2026-04-07 21:48:16 +0000 UTC,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:,ReportingInstance:,}
DEBUG: 2026/04/07 22:49:16 nodescheduler.go:541: event panda-lidar-sched-5563.18a4316ce62ad719: Failed, Error: ImagePullBackOff

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions