Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

panic: interface conversion: runtime.Object is nil, not *v1.Pod #9510

Closed
mjhuber opened this issue Aug 27, 2024 · 0 comments · Fixed by #9511
Closed

panic: interface conversion: runtime.Object is nil, not *v1.Pod #9510

mjhuber opened this issue Aug 27, 2024 · 0 comments · Fixed by #9511

Comments

@mjhuber
Copy link

mjhuber commented Aug 27, 2024

Sporadically receive the error panic: interface conversion: runtime.Object is nil, not *v1.Pod when running skaffold build.

Expected behavior

  1. Run skaffold build
  2. Container is built and pushed.

Actual behavior

  1. Run skaffold build.
  2. Approximately 5% of the time, skaffold fails, e.g.,
Building [us-docker.pkg.dev/recharge-infra-services-8b42/gcr.io/internal-admin-tools]...
panic: interface conversion: runtime.Object is nil, not *v1.Pod
goroutine 200 [running]:
github.com/GoogleContainerTools/skaffold/v2/pkg/skaffold/kubernetes.WaitForPodInitialized.func1(0xc0015cee98?)
	/tmpfs/src/git/skaffold-mirror/pkg/skaffold/kubernetes/wait.go:111 +0xf2
github.com/GoogleContainerTools/skaffold/v2/pkg/skaffold/kubernetes.watchUntilTimeout({0x3e28b30?, 0xc000b78600?}, 0x0?, {0x3e0c7c8, 0xc00137c280}, 0xc0015cefd0)
	/tmpfs/src/git/skaffold-mirror/pkg/skaffold/kubernetes/wait.go:49 +0x170
github.com/GoogleContainerTools/skaffold/v2/pkg/skaffold/kubernetes.WaitForPodInitialized({0x3e28b30, 0xc000b78600}, {0x3e432b0, 0xc0000dd020}, {0xc001162cf0, 0xc})
	/tmpfs/src/git/skaffold-mirror/pkg/skaffold/kubernetes/wait.go:110 +0x236
github.com/GoogleContainerTools/skaffold/v2/pkg/skaffold/build/cluster.(*Builder).setupKanikoBuildContext(0xc0006185b0, {0x3e28b30, 0xc000b78600}, {0x5589a70, 0x1}, {0xc0009b4af0, 0x4a}, 0xc0008143c0, {0x3e432b0, 0xc0000dd020}, ...)
	/tmpfs/src/git/skaffold-mirror/pkg/skaffold/build/cluster/kaniko.go:169 +0x88
github.com/GoogleContainerTools/skaffold/v2/pkg/skaffold/build/cluster.(*Builder).buildWithKaniko(0xc0006185b0, {0x3e28b30, 0xc000b78600}, {0x3df6c80, 0xc0005ee780}, {0x5589a70, 0x1}, {0xc0009b4af0, 0x4a}, 0xc0008143c0, ...)
	/tmpfs/src/git/skaffold-mirror/pkg/skaffold/build/cluster/kaniko.go:104 +0x9e5
github.com/GoogleContainerTools/skaffold/v2/pkg/skaffold/build/cluster.(*Builder).runBuildForArtifact(0xc0006185b0, {0x3e28b30, 0xc000b78600}, {0x3df6c80, 0xc0005ee780}, 0xc001264820, {0xc000c64080, 0x73}, {0x0, {0x0, ...}})
	/tmpfs/src/git/skaffold-mirror/pkg/skaffold/build/cluster/cluster.go:88 +0x3d1
github.com/GoogleContainerTools/skaffold/v2/pkg/skaffold/build/cluster.(*Builder).buildArtifact(0xc0006185b0, {0x3e28b30?, 0xc000b78600?}, {0x3df6c80, 0xc0005ee780}, 0xc001264820, {0xc000c64080, 0x73}, {0x0, {0x0, ...}})
	/tmpfs/src/git/skaffold-mirror/pkg/skaffold/build/cluster/cluster.go:71 +0x146
github.com/GoogleContainerTools/skaffold/v2/pkg/skaffold/build.(*BuilderMux).Build.func1({0x3e28b30, 0xc000b78600}, {0x3df6c80, 0xc0005ee780}, 0xc001264820, {0xc000c64080, 0x73}, {0x0, {0x0, 0x0, ...}})
	/tmpfs/src/git/skaffold-mirror/pkg/skaffold/build/builder_mux.go:112 +0x61d
github.com/GoogleContainerTools/skaffold/v2/pkg/skaffold/build.performBuild({0x3e28b30, 0xc000b78600}, {0x3df6c80, 0xc0005ee780}, 0x11?, {0xc0007a7d80?}, 0xc001264820, 0xc001d94b40)
	/tmpfs/src/git/skaffold-mirror/pkg/skaffold/build/scheduler.go:170 +0x213
github.com/GoogleContainerTools/skaffold/v2/pkg/skaffold/build.(*scheduler).build(0xc000370d90, {0x3e28b68, 0xc0005ee730}, 0xc001209080, {0xc0003ec738?}, 0x0)
	/tmpfs/src/git/skaffold-mirror/pkg/skaffold/build/scheduler.go:121 +0x70b
github.com/GoogleContainerTools/skaffold/v2/pkg/skaffold/build.(*scheduler).run.func1()
	/tmpfs/src/git/skaffold-mirror/pkg/skaffold/build/scheduler.go:77 +0x2b
golang.org/x/sync/errgroup.(*Group).Go.func1()
	/tmpfs/src/git/skaffold-mirror/vendor/golang.org/x/sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1
	/tmpfs/src/git/skaffold-mirror/vendor/golang.org/x/sync/errgroup/errgroup.go:75 +0x96

Information

  • Skaffold version: v2.13.1
  • Operating system: Alpine Linux v3.18
  • Installed via: https://storage.googleapis.com/skaffold/releases/v2.13.1/skaffold-linux-amd64
---
apiVersion: skaffold/v2beta20
kind: Config
metadata:
  name: internal-admin-tools
build:
  tagPolicy:
    gitCommit:
      variant: CommitSha
  cluster:
    namespace: gitlab
    volumes:
      - name: kaniko-secret
        secret:
          secretName: kaniko-secret
  artifacts:
    - context: .
      image: us-docker.pkg.dev/recharge-infra-services-8b42/gcr.io/internal-admin-tools
      kaniko:
        verbosity: debug
        dockerfile: Dockerfile
        skipUnusedStages: true
        useNewRun: true
        env:
          - name: GOOGLE_APPLICATION_CREDENTIALS
            value: /kaniko-secret/kaniko-secret.json
        cache:
          ttl: 336h
        volumeMounts:
          - name: kaniko-secret
            mountpath: /kaniko-secret
            readonly: true
        registryMirror: mirror.gcr.io
deploy:
  kubeContext: <redacted>
  statusCheckDeadlineSeconds: 300
  helm:
    flags:
      upgrade: [--timeout, 300s]
      install: [--timeout, 300s]
    releases:
      - name: internal-admin-tools
        chartPath: deploy/charts/internal-admin-tools
        namespace: default
        createNamespace: true
        upgradeOnChange: true
        wait: true
        valuesFiles:
          - deploy/local/values.yml
        artifactOverrides:
          image: us-docker.pkg.dev/recharge-infra-services-8b42/gcr.io/internal-admin-tools
profiles:
  - name: stage
    patches:
      - op: replace
        path: /deploy/helm/releases/0/namespace
        value: internal-admin-tools
      - op: replace
        path: /deploy/helm/releases/0/valuesFiles/0
        value: deploy/stage/values.yml
  - name: prod
    patches:
      - op: replace
        path: /deploy/helm/releases/0/namespace
        value: internal-admin-tools
      - op: replace
        path: /deploy/helm/releases/0/valuesFiles/0
        value: deploy/prod/values.yml

Steps to reproduce the behavior

Issue is very sporadic, running skaffold build many times will yield this result occasionally.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(wait): Add panic prevent WaitForPodInitialized idsulik/skaffold
1 participant
@mjhuber