Skip to content

fix: ignore internal configs in config hash calculation - #8868

Merged
leonardoce merged 2 commits into
mainfrom
dev/8371
Oct 20, 2025
Merged

leonardoce merged 2 commits into
mainfrom
dev/8371

Conversation

@mnencia

@mnencia mnencia commented Oct 15, 2025

Copy link
Copy Markdown
Member

Closes #8371

@mnencia
mnencia requested a review from a team October 15, 2025 13:16
@cnpg-bot cnpg-bot added backport-requested ◀️ This pull request should be backported to all supported releases release-1.25 labels Oct 15, 2025
@github-actions

Copy link
Copy Markdown
Contributor

❗ By default, the pull request is configured to backport to all release branches.

  • To stop backporting this pr, remove the label: backport-requested ◀️ or add the label 'do not backport'
  • To stop backporting this pr to a certain release branch, remove the specific branch label: release-x.y

@dosubot dosubot Bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Oct 15, 2025
@dosubot dosubot Bot added the bug 🐛 Something isn't working label Oct 15, 2025
@leonardoce

Copy link
Copy Markdown
Contributor

/test

@github-actions

Copy link
Copy Markdown
Contributor

@armru

armru commented Oct 16, 2025

Copy link
Copy Markdown
Member

/test limit=local

@github-actions

Copy link
Copy Markdown
Contributor

@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label Oct 16, 2025
@cnpg-bot cnpg-bot added the ok to merge 👌 This PR can be merged label Oct 16, 2025
mnencia and others added 2 commits October 20, 2025 15:21
@leonardoce
leonardoce merged commit 076273e into main Oct 20, 2025
3 checks passed
@leonardoce
leonardoce deleted the dev/8371 branch October 20, 2025 13:34
cnpg-bot pushed a commit that referenced this pull request Oct 20, 2025
Closes #8371

Signed-off-by: Marco Nenciarini <[email protected]>
Signed-off-by: Armando Ruocco <[email protected]>
Co-authored-by: Armando Ruocco <[email protected]>
(cherry picked from commit 076273e)
mnencia added a commit that referenced this pull request Mar 22, 2026
…es (#10342)

When upgrading from an operator version that includes cnpg.* parameters
in the PostgreSQL configuration hash to one that excludes them (change
introduced in #8868), pods running the old and new operator end up
computing different hashes for the same effective configuration. The
uniformity check in reconcilePods then blocks indefinitely, waiting for
all pods to report the same hash, but that can never happen because the
remaining pods still use the old algorithm. Neither the rolling update
nor the in-place instance manager upgrade can run, since both live
behind the blocked check.

The fix introduces `isConfigNonUniformityFromUpgrade()`, which groups
pods by their operator version and checks whether the hash split aligns
exactly with the version split. If every pod running the same operator
version agrees on its hash, but different versions disagree, the
non-uniformity is structural and will never self-resolve. In that case
we skip the wait and let `handleRollingUpdate` proceed.

The operator version key combines both the bootstrap controller image
(changes on rolling updates) and the executable hash (changes on
in-place upgrades), so the deadlock is detected regardless of the
upgrade strategy.

When the non-uniformity is from normal config propagation (pods on the
same operator version reporting different hashes), the function returns
false and the existing wait-for-uniformity behavior is preserved. This
keeps the fix scoped strictly to the operator upgrade deadlock scenario,
with no change in behavior during normal operations.

Closes: #10348

Assisted-by: Claude
Signed-off-by: Armando Ruocco <[email protected]>
Signed-off-by: Marco Nenciarini <[email protected]>
Co-authored-by: Marco Nenciarini <[email protected]>
cnpg-bot pushed a commit that referenced this pull request Mar 22, 2026
…es (#10342)

When upgrading from an operator version that includes cnpg.* parameters
in the PostgreSQL configuration hash to one that excludes them (change
introduced in #8868), pods running the old and new operator end up
computing different hashes for the same effective configuration. The
uniformity check in reconcilePods then blocks indefinitely, waiting for
all pods to report the same hash, but that can never happen because the
remaining pods still use the old algorithm. Neither the rolling update
nor the in-place instance manager upgrade can run, since both live
behind the blocked check.

The fix introduces `isConfigNonUniformityFromUpgrade()`, which groups
pods by their operator version and checks whether the hash split aligns
exactly with the version split. If every pod running the same operator
version agrees on its hash, but different versions disagree, the
non-uniformity is structural and will never self-resolve. In that case
we skip the wait and let `handleRollingUpdate` proceed.

The operator version key combines both the bootstrap controller image
(changes on rolling updates) and the executable hash (changes on
in-place upgrades), so the deadlock is detected regardless of the
upgrade strategy.

When the non-uniformity is from normal config propagation (pods on the
same operator version reporting different hashes), the function returns
false and the existing wait-for-uniformity behavior is preserved. This
keeps the fix scoped strictly to the operator upgrade deadlock scenario,
with no change in behavior during normal operations.

Closes: #10348

Assisted-by: Claude
Signed-off-by: Armando Ruocco <[email protected]>
Signed-off-by: Marco Nenciarini <[email protected]>
Co-authored-by: Marco Nenciarini <[email protected]>
(cherry picked from commit 324709c)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-requested ◀️ This pull request should be backported to all supported releases bug 🐛 Something isn't working lgtm This PR has been approved by a maintainer ok to merge 👌 This PR can be merged size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: CNPG 1.26 -> 1.27 migration stuck

4 participants