Skip to content

Unable to create backup, panic: assignment to entry in nil map #523

Closed

Description

@songtianlun

I followed this guide https://cloudnative-pg.io/plugin-barman-cloud/docs/usage/ to create a backup, but immediately encountered this error. I tried deploying both version 0.6.0 and the latest main branch, but both resulted in this error. Could you help me identify the cause?

➜  umami git:(main) kubectl -n umami describe backup backup-example
Name:         backup-example
Namespace:    umami
Labels:       <none>
Annotations:  <none>
API Version:  postgresql.cnpg.io/v1
Kind:         Backup
Metadata:
  Creation Timestamp:  2025-09-17T15:52:13Z
  Generation:          1
  Resource Version:    93357031
  UID:                 f00d6637-465b-4d13-afa5-4a9320a14f91
Spec:
  Cluster:
    Name:  umami-pg-cluster
  Method:  plugin
  Plugin Configuration:
    Name:  barman-cloud.cloudnative-pg.io
Status:
  Error:  rpc error: code = Unknown desc = panic caught: assignment to entry in nil map

goroutine 354 [running]:
github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors/recovery.recoverFrom({0x2718080?, 0xc00002aab0?}, {0x1fe6fc0, 0x3a52170}, 0xc0005f67a7?)
  /go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/[email protected]/interceptors/recovery/interceptors.go:57 +0x65
github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors/recovery.UnaryServerInterceptor.func1.1()
  /go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/[email protected]/interceptors/recovery/interceptors.go:30 +0x67
panic({0x1fe6fc0?, 0x3a52170?})
  /usr/local/go/src/runtime/panic.go:783 +0x132
k8s.io/apimachinery/pkg/util/runtime.handleCrash({0x2718160, 0x3ac5900}, {0x1fe6fc0, 0x3a52170}, {0xc0005f6988, 0x0, 0xc000983300?})
  /go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:114 +0x1a9
k8s.io/apimachinery/pkg/util/runtime.HandleCrash({0x0, 0x0, 0xc000794c40?})
  /go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:64 +0x17b
panic({0x1fe6fc0?, 0x3a52170?})
  /usr/local/go/src/runtime/panic.go:783 +0x132
k8s.io/apimachinery/pkg/util/runtime.handleCrash({0x2718160, 0x3ac5900}, {0x1fe6fc0, 0x3a52170}, {0xc0005f6b70, 0x0, 0x2?})
  /go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:114 +0x1a9
k8s.io/apimachinery/pkg/util/runtime.HandleCrash({0x0, 0x0, 0xc000794c40?})
  /go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:64 +0x17b
panic({0x1fe6fc0?, 0x3a52170?})
  /usr/local/go/src/runtime/panic.go:783 +0x132
github.com/cloudnative-pg/plugin-barman-cloud/internal/cnpgi/instance.setLastFailedBackupTime.func1()
  /workspace/internal/cnpgi/instance/recovery_window.go:62 +0x1e5
k8s.io/client-go/util/retry.OnError.func1()
  /go/pkg/mod/k8s.io/[email protected]/util/retry/util.go:51 +0x30
k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection(0x493caa?)
  /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:150 +0x3e
k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff({0x989680, 0x4014000000000000, 0x3fb999999999999a, 0x4, 0x0}, 0xc0005f6f08)
  /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/backoff.go:477 +0x5a
k8s.io/client-go/util/retry.OnError({0x989680, 0x4014000000000000, 0x3fb999999999999a, 0x4, 0x0}, 0xc000982d00?, 0xc0005f6f90?)
  /go/pkg/mod/k8s.io/[email protected]/util/retry/util.go:50 +0x96
k8s.io/client-go/util/retry.RetryOnConflict(...)
  /go/pkg/mod/k8s.io/[email protected]/util/retry/util.go:104
github.com/cloudnative-pg/plugin-barman-cloud/internal/cnpgi/instance.setLastFailedBackupTime({0x2718080?, 0xc00002aab0?}, {0x272c460?, 0xc0005dff50?}, {{0xc0009740d7?, 0xc0005f70c8?}, {0xc0009742c0?, 0x0?}}, {0xc0009740e0, 0x10}, ...)
  /workspace/internal/cnpgi/instance/recovery_window.go:54 +0x152
github.com/cloudnative-pg/plugin-barman-cloud/internal/cnpgi/instance.BackupServiceImplementation.handleBackupError.func1()
  /workspace/internal/cnpgi/instance/backup.go:182 +0xdd
k8s.io/client-go/util/retry.OnError.func1()
  /go/pkg/mod/k8s.io/[email protected]/util/retry/util.go:51 +0x30
k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection(0x1?)
  /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:150 +0x3e
k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff({0x989680, 0x4014000000000000, 0x3fb999999999999a, 0x4, 0x0}, 0xc0005f71a8)
  /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/backoff.go:477 +0x5a
k8s.io/client-go/util/retry.OnError({0x989680, 0x4014000000000000, 0x3fb999999999999a, 0x4, 0x0}, 0x0?, 0x0?)
  /go/pkg/mod/k8s.io/[email protected]/util/retry/util.go:50 +0x96
k8s.io/client-go/util/retry.RetryOnConflict(...)
  /go/pkg/mod/k8s.io/[email protected]/util/retry/util.go:104
github.com/cloudnative-pg/plugin-barman-cloud/internal/cnpgi/instance.BackupServiceImplementation.handleBackupError({{0x272c460, 0xc0005dff50}, {0xc000054029, 0x12}, {}}, {0x2718080?, 0xc00002aab0?}, 0x0?)
  /workspace/internal/cnpgi/instance/backup.go:179 +0x109
github.com/cloudnative-pg/plugin-barman-cloud/internal/cnpgi/instance.BackupServiceImplementation.Backup({{0x272c460, 0xc0005dff50}, {0xc000054029, 0x12}, {}}, {0x2718080, 0xc00002aab0}, 0xc0009881e0)
  /workspace/internal/cnpgi/instance/backup.go:106 +0x65a
github.com/cloudnative-pg/cnpg-i/pkg/backup._Backup_Backup_Handler.func1({0x2718080?, 0xc00002aab0?}, {0x2191fc0?, 0xc0009881e0?})
  /go/pkg/mod/github.com/cloudnative-pg/[email protected]/pkg/backup/backup_grpc.pb.go:140 +0xcb
github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors/recovery.UnaryServerInterceptor.func1({0x2718080?, 0xc00002aab0?}, {0x2191fc0?, 0xc0009881e0?}, 0x1fe7320?, 0x3ac5900?)
  /go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/[email protected]/interceptors/recovery/interceptors.go:34 +0x8a
google.golang.org/grpc.getChainUnaryHandler.func1.getChainUnaryHandler.1({0x2718080, 0xc00002aab0}, {0x2191fc0, 0xc0009881e0})
  /go/pkg/mod/google.golang.org/[email protected]/server.go:1243 +0xa8
github.com/cloudnative-pg/cnpg-i-machinery/pkg/pluginhelper/http.(*Server).Start.loggingUnaryServerInterceptor.func3({0x2718080?, 0xc00002aa80?}, {0x2191fc0, 0xc0009881e0}, 0x189ecda?, 0xc000982640)
  /go/pkg/mod/github.com/cloudnative-pg/[email protected]/pkg/pluginhelper/http/grpc.go:37 +0x48
google.golang.org/grpc.getChainUnaryHandler.func1({0x2718080?, 0xc00002aa80?}, {0x2191fc0?, 0xc0009881e0?})
  /go/pkg/mod/google.golang.org/[email protected]/server.go:1243 +0x153
github.com/cloudnative-pg/cnpg-i-machinery/pkg/pluginhelper/http.(*Server).Start.logFailedRequestsUnaryServerInterceptor.func2({0x2718080?, 0xc00002aa80?}, {0x2191fc0?, 0xc0009881e0?}, 0xc0008202c0, 0xc000020480?)
  /go/pkg/mod/github.com/cloudnative-pg/[email protected]/pkg/pluginhelper/http/grpc.go:49 +0x54
google.golang.org/grpc.NewServer.chainUnaryServerInterceptors.chainUnaryInterceptors.func1({0x2718080, 0xc00002aa80}, {0x2191fc0, 0xc0009881e0}, 0xc0008202c0, 0x80?)
  /go/pkg/mod/google.golang.org/[email protected]/server.go:1234 +0x7c
github.com/cloudnative-pg/cnpg-i/pkg/backup._Backup_Backup_Handler({0x21ed8a0, 0xc0004a56e0}, {0x2718080, 0xc00002aa80}, 0xc000984400, 0xc0004a7d60)
  /go/pkg/mod/github.com/cloudnative-pg/[email protected]/pkg/backup/backup_grpc.pb.go:142 +0x143
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0001a6a00, {0x2718080, 0xc00002a9f0}, 0xc000988180, 0xc00062a060, 0x3a73b58, 0x0)
  /go/pkg/mod/google.golang.org/[email protected]/server.go:1431 +0xfe8
google.golang.org/grpc.(*Server).handleStream(0xc0001a6a00, {0x27189c0, 0xc0004641a0}, 0xc000988180)
  /go/pkg/mod/google.golang.org/[email protected]/server.go:1842 +0xb95
google.golang.org/grpc.(*Server).serveStreams.func2.1()
  /go/pkg/mod/google.golang.org/[email protected]/server.go:1061 +0x7f
created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 334
  /go/pkg/mod/google.golang.org/[email protected]/server.go:1072 +0x11d

  Instance ID:
    Container ID:  containerd://01afa109471c0e4714c3e98d98a27d48c7474f1c78f539cb3312600a28f253fc
    Pod Name:      umami-pg-cluster-1
  Method:          plugin
  Phase:           failed
Events:
  Type    Reason    Age   From                   Message
  ----    ------    ----  ----                   -------
  Normal  Starting  5s    cloudnative-pg-backup  Starting backup for cluster umami-pg-cluster
  Normal  Starting  4s    local-webserver        Backup started
  Normal  Failed    0s    local-webserver        Backup failed

Activity

  1. dosubot commented on Sep 17, 2025

    @dosubot

    Hi @songtianlun! I'm Dosu and I’m helping the plugin-barman-cloud team.

    This error is caused by a bug in the plugin-barman-cloud code: the function that updates the backup status (setLastFailedBackupTime) tries to write to a map (ServerRecoveryWindow) that hasn't been initialized yet, which triggers a panic in Go when the backup fails. This is not a configuration or usage mistake—it's a missing map initialization in the plugin itself, and it affects both v0.6.0 and the latest main branch. There is currently no workaround or configuration change that can prevent this; a code fix is required in the plugin to properly initialize the map before assignment, as is done elsewhere in the code. You can track this issue and monitor for a fix in the plugin-barman-cloud repository. More details on the root cause are available here.

    To reply, just mention @dosu.


    Share context across your team and agents. Try Dosu.

    Leave Feedback Learn about plugin-barman-cloud with Dosu Add Dosu to your team

  2. songtianlun commented on Sep 18, 2025

    @songtianlun
    Author

    It might be because I'm using Cloudflare R2 storage. While I've been able to seamlessly replace Minio/AWS S3 with R2 elsewhere, it doesn't work here. I can't identify the issue at the moment, so I'll switch back to Minio for now.

  3. leonardoce commented on Sep 18, 2025

    @leonardoce
    Contributor

    It might be, but the code should definitely not panic. I'm proposing a fix.

  4. added 5 commits that reference this issue on Sep 18, 2025
    bbbc546
    d116bf8
    bda7310
    f89982d
    543bc16
  5. added a commit that references this issue on Sep 18, 2025
    7653780
  6. added a commit that references this issue on Sep 18, 2025
    dfd9861
  7. added 3 commits that reference this issue on Oct 6, 2025
    9e3ccac
    5612fad
    9363650
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions