Skip to content

[Bug]: websocket exec fails with 403 Forbidden on OpenShift (regression from #10876) #10931

Description

@mnencia

Is there an existing issue already for this bug?

  • I have searched for an existing issue, and could not find anything. I believe this is a new bug.

I have read the troubleshooting guide

  • I have read the troubleshooting guide and I think this is a new bug.

I am running a supported version of CloudNativePG

  • I have read the troubleshooting guide and I think this is a new bug.

Contact Details

[email protected]

Version

trunk (main)

What version of Kubernetes are you using?

1.31

What is your Kubernetes environment?

Other

How did you install the operator?

YAML manifest

What happened?

PR #10876 changed execCommandOnce in pkg/utils/exec.go from remotecommand.NewSPDYExecutor to remotecommand.NewWebSocketExecutor. On OpenShift (all tested versions: 4.18–4.21, corresponding to Kubernetes 1.31–1.34), the WebSocket exec upgrade is rejected with 403 Forbidden, breaking every exec call through that path.

This was caught in pg4k nightly E2E on 2026-06-17, the first nightly to include the commit. All 4 OpenShift jobs accumulated ~200 instances of this error each. On OCP 4.18 the failures surface immediately; on 4.19–4.21 they appear as 600s Eventually timeouts because the error fires on every poll attempt inside the retry loop.

The fix is to use NewFallbackExecutor (available in client-go v0.32+), which tries WebSocket first and falls back to SPDY only when the handshake fails. This preserves the fix for vanilla Kubernetes 1.31+ (the original #10873) while restoring compatibility with OpenShift.

wsExecutor, err := remotecommand.NewWebSocketExecutor(&newConfig, "POST", req.URL().String())
if err != nil {
    return "", "", err
}
spdyExecutor, err := remotecommand.NewSPDYExecutor(&newConfig, "POST", req.URL())
if err != nil {
    return "", "", err
}
executor, err := remotecommand.NewFallbackExecutor(wsExecutor, spdyExecutor, func(err error) bool {
    return strings.Contains(err.Error(), "websocket: bad handshake")
})
if err != nil {
    return "", "", err
}

Cluster resource

Relevant log output

error: unable to upgrade streaming request: websocket: bad handshake (403 Forbidden)
cmd: [psql -U postgres postgres -tAc select datcollate from pg_catalog.pg_database where datname='template0']
error: unable to upgrade streaming request: websocket: bad handshake (403 Forbidden)

Code of Conduct

  • I agree to follow this project's Code of Conduct

Activity

  1. self-assigned this
    on Jun 17, 2026
  2. added theissue type on Jun 17, 2026
  3. added this to the 1.30.0 milestone on Jun 18, 2026
  4. moved this to Waiting for First Review in CloudNativePG operator developmenton Jun 18, 2026
  5. moved this from Waiting for First Review to Done in CloudNativePG operator developmenton Jun 18, 2026
  6. added a commit that references this issue on Jun 18, 2026
    f6c106c
  7. added a commit that references this issue on Jun 18, 2026
    73888b6
  8. added
    bug 🐛Something isn't working
    and removed on Jun 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bug 🐛Something isn't working

Type

Projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions