Repository navigation
fix(postgres): render the declared databases in a stable order - #5091
Aleksei Sviridkin (lexfrei) merged 1 commit into
Conversation
keys returns a map's keys in Go's randomized iteration order, so a release with several databases rendered DEFINED_DBS in a different order on each upgrade. The init-script Secret, and the checksum the init Job carries, then changed on upgrades that left the databases untouched, which makes a no-op upgrade look like a real change in the release history. Assisted-by: LLM Signed-off-by: Mattia Eleuteri <[email protected]>
|
No actionable comments were generated in the recent review.
|
| Layer / File(s) | Summary |
|---|---|
Sort and test database names packages/apps/postgres/templates/init-script.yaml, packages/apps/postgres/tests/init_script_deterministic_test.yaml |
The init script sorts database keys before joining them into DEFINED_DBS. The Helm test verifies alphabetical output for eight databases declared in a different order. |
Priority:
Estimated code review effort: 2 (Simple) | ~8 minutes
Change: Bug fix · Severity of issue fixed: Low
Possibly related issues
cozystack/cozystack#5092— The issue describes unstable database ordering in manifests during no-op upgrades. This change sorts the database keys used in the init script.
Suggested reviewers: kvaps
Merge Risk: 
Minimal · up to dc946
-
No issue identified that should prevent merging after normal checks.
Pre-merge checks: 
5
-
Full details: Docstring Coverage
-
Explanation
No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (2 skipped: 2 unsupported.)
-
Finishing Touches
-
🧪 Generate unit tests (beta)
-
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment @coderabbitai help to get the list of available commands.
Aleksei Sviridkin (lexfrei)
left a comment
There was a problem hiding this comment.
mattia-eleuteri LGTM. In fixtures without generated user credentials, repeated full renders stay identical for unchanged database maps, including the init Job checksum. I checked empty and single-entry maps, mixed-case names and punctuation, plus exact-name membership in the deletion list. Removing the sort makes the regression test fail; restoring it passes, and the full chart suite passes with 210 tests.
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->
## What this PR does
`databases` accepts `postgres`, `template0` and `template1`, and the
init job treats them like application databases: it comments them
`database managed by helm`, creates `<db>_admin` and `<db>_readonly`,
makes `<db>_admin` the owner of the database and its schemas, and
installs the `trigger_auto_grant` event trigger. When the entry later
leaves values, the delete pass runs `DROP DATABASE` on it, which can
only fail:
- `template1`, `template0`: `cannot drop a template database`;
- `postgres`: `cannot drop the currently open database`, since the job
is connected to it.
The init job then fails on every upgrade, so the HelmRelease sits in
`UpgradeFailed` with `post-upgrade hooks failed`, and putting the entry
back does not help. `template1` also hands the takeover (owner of
`public`, default privileges, trigger) to every database created
afterwards. Once its roles are dropped by hand, the trigger left behind
names a role that no longer exists, so `CREATE SCHEMA` fails in every
new database, even as superuser.
This PR:
- **Refuses** the three names at render with a message that says what to
do. `cozyvalues-gen` has no way to constrain the keys of a
`map[string]Database`, so the refusal sits next to the existing
database-name guard, like the `users.postgres` refusal. The field
description says it too.
- **Never selects them for `DROP DATABASE`.** The delete pass excludes
them whatever values say.
- **Hands a taken-over system database back.** In a new `release system
databases` pass before the delete pass, for each of the three still
commented `database managed by helm`: drop the trigger and
`auto_grant_schema_privileges()`, `REASSIGN OWNED` / `DROP OWNED` for
its two roles in that database, give `public` back to
`pg_database_owner` (PostgreSQL 15+), and clear the comment. The
existing orphaned-roles pass then drops the roles; it never visits
`template1`, hence the per-database step. `template0` accepts no
connection and the job that commented it never got further, so only its
comment is cleared.
**Upgrade note.** A release that declares one of the three names fails
to render until the entry is removed. Removing it is now safe: the next
run releases the database instead of dropping it.
**Testing**
- `tests/system_databases_test.yaml`: 5 of its 6 cases fail on `main`
without the change. `helm unittest packages/apps/postgres`: 215 passed.
- I replayed the scenario against a `postgres:17` container, running the
rendered `init.sh` with `PGUSER=postgres`:
1. The `main` script with `databases: {appdb, template1, postgres}`
reproduces the takeover.
2. The `main` script with only `appdb` then fails with `cannot drop the
currently open database`.
3. This branch's script on that same state exits 0. It releases both
databases (owner `postgres`, no comment, no event trigger, `public`
owned by `pg_database_owner`) and drops `postgres_admin`,
`postgres_readonly`, `template1_admin` and `template1_readonly`. A
database created afterwards gets no trigger, and `CREATE SCHEMA`
succeeds in it.
4. A second run is a no-op.
5. A `template0` commented and given roles by hand is released the same
way.
Open PR #5091 touches the same script (stable database order); the hunks
do not overlap.
Backport: the automatic cherry-pick onto `release-1.6` conflicts in
`init-script.yaml` and the generated cozyrd, so the backport to
`release-1.6` is done by hand in #5157 (same SQL, replayed against that
branch's script). `release-1.7` keeps `kind/backport`.
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Downstream repositories
<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.
Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.
IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:
1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->
- [x] No downstream repository is affected by this change (walked the
map: a template guard, the init script, a test suite and one field
description regenerated into README, schema, Go types and cozyrd; no
field, default, enum or name change)
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
fix(postgres): databases now refuses the PostgreSQL system databases postgres, template0 and template1, and the init job never runs DROP DATABASE on them. A system database a previous version took over is handed back to the superuser on the next upgrade (event trigger removed, ownership and public schema restored, its <db>_admin and <db>_readonly roles dropped). A release that declares one of them must remove the entry before upgrading.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Behavior Changes**
* PostgreSQL system databases (`postgres`, `template0`, and `template1`)
are refused as managed application databases.
* Previously managed system databases are released during cleanup and
excluded from database deletion. Cleanup restores eligible ownership and
security settings and clears system database comments.
* **Documentation**
* Updated PostgreSQL configuration guidance to clarify that these system
database names are refused. Names that merely contain these strings
remain accepted.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does The postgres init script lists the declared databases with `keys .Values.databases`, which returns them in Go's randomized map order. A release with two or more databases therefore renders a different `DEFINED_DBS` line on each upgrade, so the `<release>-init-script` Secret and the `checksum/config` annotation of the init Job change on upgrades that did not touch the databases. Sorting the keys makes the render a function of the values alone. Found while measuring a platform-wide values change on a production cluster (#5092): of 36 releases that ran Jobs again, 31 rendered a byte-identical manifest across the two Helm revisions and the other 5 were Postgres releases whose only difference was this ordering, e.g. `DEFINED_DBS="db_a db_b"` → `DEFINED_DBS="db_b db_a"`. This does not stop the init Job from running on every upgrade (it is a `post-upgrade` hook); the issue covers that part. Verification: the new `tests/init_script_deterministic_test.yaml` renders eight databases declared out of order and expects them sorted. Without the fix it failed 5 runs out of 5 with a different order each time; with it, `helm unittest packages/apps/postgres` passes (27 suites, 210 tests). ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(postgres): the init script renders the declared databases in a stable order, so an upgrade that leaves the databases unchanged no longer rewrites the init-script Secret. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **Bug Fixes** * Database initialization now lists configured databases in alphabetical order, providing consistent output across deployments. * **Tests** * Added coverage to verify the alphabetical ordering. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Backported-from: 0c48626
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `kind/backport` if it's a bugfix that needs to be
backported to a previous version.
-->
## What this PR does
`databases` accepts `postgres`, `template0` and `template1`, and the
init job treats them like application databases: it comments them
`database managed by helm`, creates `<db>_admin` and `<db>_readonly`,
makes `<db>_admin` the owner of the database and its schemas, and
installs the `trigger_auto_grant` event trigger. When the entry later
leaves values, the delete pass runs `DROP DATABASE` on it, which can
only fail:
- `template1`, `template0`: `cannot drop a template database`;
- `postgres`: `cannot drop the currently open database`, since the job
is connected to it.
The init job then fails on every upgrade, so the HelmRelease sits in
`UpgradeFailed` with `post-upgrade hooks failed`, and putting the entry
back does not help. `template1` also hands the takeover (owner of
`public`, default privileges, trigger) to every database created
afterwards. Once its roles are dropped by hand, the trigger left behind
names a role that no longer exists, so `CREATE SCHEMA` fails in every
new database, even as superuser.
This PR:
- **Refuses** the three names at render with a message that says what to
do. `cozyvalues-gen` has no way to constrain the keys of a
`map[string]Database`, so the refusal sits next to the existing
database-name guard, like the `users.postgres` refusal. The field
description says it too.
- **Never selects them for `DROP DATABASE`.** The delete pass excludes
them whatever values say.
- **Hands a taken-over system database back.** In a new `release system
databases` pass before the delete pass, for each of the three still
commented `database managed by helm`: drop the trigger and
`auto_grant_schema_privileges()`, `REASSIGN OWNED` / `DROP OWNED` for
its two roles in that database, give `public` back to
`pg_database_owner` (PostgreSQL 15+), and clear the comment. The
existing orphaned-roles pass then drops the roles; it never visits
`template1`, hence the per-database step. `template0` accepts no
connection and the job that commented it never got further, so only its
comment is cleared.
**Upgrade note.** A release that declares one of the three names fails
to render until the entry is removed. Removing it is now safe: the next
run releases the database instead of dropping it.
**Testing**
- `tests/system_databases_test.yaml`: 5 of its 6 cases fail on `main`
without the change. `helm unittest packages/apps/postgres`: 215 passed.
- I replayed the scenario against a `postgres:17` container, running the
rendered `init.sh` with `PGUSER=postgres`:
1. The `main` script with `databases: {appdb, template1, postgres}`
reproduces the takeover.
2. The `main` script with only `appdb` then fails with `cannot drop the
currently open database`.
3. This branch's script on that same state exits 0. It releases both
databases (owner `postgres`, no comment, no event trigger, `public`
owned by `pg_database_owner`) and drops `postgres_admin`,
`postgres_readonly`, `template1_admin` and `template1_readonly`. A
database created afterwards gets no trigger, and `CREATE SCHEMA`
succeeds in it.
4. A second run is a no-op.
5. A `template0` commented and given roles by hand is released the same
way.
Open PR #5091 touches the same script (stable database order); the hunks
do not overlap.
Backport: the automatic cherry-pick onto `release-1.6` conflicts in
`init-script.yaml` and the generated cozyrd, so the backport to
`release-1.6` is done by hand in #5157 (same SQL, replayed against that
branch's script). `release-1.7` keeps `kind/backport`.
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Downstream repositories
<!--
Cozystack is upstream for repositories that are NOT kept in sync with it
automatically, and nothing in CI compares the two sides, so a change
here can break them silently.
Which change forces what, and which file to touch, is the trigger map in
docs/agents/contributing.md under "Downstream Repositories". Walk it
before you tick anything.
IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM
TO FILL IN:
1. Walk the trigger map against the actual diff of this PR, file by
file. Do not decide from the PR title or from your memory of what you
changed.
2. For every repository you tick, open the follow-up PR there and link
it on the same line. A ticked box with no link claims work that does not
exist, which is worse than an empty box. Search that repository for an
open PR or issue covering it first, and link that instead of filing a
duplicate.
3. When the follow-up is out of scope, or needs a decision that is not
yours to make, open an issue in that repository instead, link that, and
say so in the PR body. Do not push a speculative PR into another
repository just to fill a line here.
4. Do not tick "No downstream repository is affected" to make the
checklist go away. If you are unsure, leave every box empty and say so
in the PR body, so a human decides.
-->
- [x] No downstream repository is affected by this change (walked the
map: a template guard, the init script, a test suite and one field
description regenerated into README, schema, Go types and cozyrd; no
field, default, enum or name change)
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
fix(postgres): databases now refuses the PostgreSQL system databases postgres, template0 and template1, and the init job never runs DROP DATABASE on them. A system database a previous version took over is handed back to the superuser on the next upgrade (event trigger removed, ownership and public schema restored, its <db>_admin and <db>_readonly roles dropped). A release that declares one of them must remove the entry before upgrading.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **Behavior Changes**
* PostgreSQL system databases (`postgres`, `template0`, and `template1`)
are refused as managed application databases.
* Previously managed system databases are released during cleanup and
excluded from database deletion. Cleanup restores eligible ownership and
security settings and clears system database comments.
* **Documentation**
* Updated PostgreSQL configuration guidance to clarify that these system
database names are refused. Names that merely contain these strings
remain accepted.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Backported-from: b53ca1f
## What this PR does Backports to `release-1.7` from `main`, picked by the rules of the [backport policy](https://github.com/cozystack/cozystack/blob/main/docs/release/backports.md) in merge order. A merged pull request comes as its own commits, picked with `git cherry-pick -x` so each keeps its author, message and sign-offs, and a merge commit that carries the original's merge message with a `Backported-from:` trailer naming it; a squash-merged one comes as one pick. `backport-batch verify` checks that shape. A pick listed as changed differs from the commit it came from; `git range-diff <merge>^1..<merge>^2 <batch merge>^1..<batch merge>^2` shows how. A conflict or a failing check here is a [backport task](https://github.com/cozystack/cozystack/blob/main/docs/release/backport-tasks/README.md). - [#4756](#4756) `fix(backupstrategy-controller): run backup clients on a package-owned image`, 2 commits, changed in the pick: `48d0cb2958a5` - [#4867](#4867) `fix(e2e): keep chainsaw asserts polling through transient API errors`, 1 commit - [#4888](#4888) `fix(ci): dispatch frozen-line builds with GITHUB_TOKEN`, 1 commit - [#4985](#4985) `fix(etcd): stop counting canceled watch streams as failed gRPC requests`, 1 commit - [#5005](#5005) `fix(tcp-balancer): cap HAProxy fds and threads by the pod's own limits`, 1 commit - [#4917](#4917) `fix(kubernetes): run Proxmox CCM v0.16.1 and pin the Proxmox images by digest`, 2 commits - [#4974](#4974) `fix(kubernetes-nodes): plan autoscaler capacity on the cores the VM actually gets`, 1 commit - [#4971](#4971) `fix(e2e): keep the Proxmox tenant kubeconfig per run and remove it`, 2 commits - [#4970](#4970) `fix(e2e): read the KubernetesNodes pool release in the HelmRelease guard`, 3 commits - [#4972](#4972) `fix(e2e): validate the leak allowlist against every suite layout`, 3 commits - [#5082](#5082) `fix(marketplace): pin a tapped repository to a digest instead of a mutable tag`, 15 commits, changed in the pick: `a31ae9090928`, `b326e0badd62`, `a3d883b4bf5b`, `f8509450f7ca` - [#4004](#4004) `fix(dashboard): link a tenant row to its detail page`, 2 commits - [#5212](#5212) `fix(cilium): pass the CiliumNode list to jq on stdin in the IPAM hook`, 1 commit - [#5213](#5213) `fix(kubernetes): move the OIDC users out of the bootstrap Job's script`, 1 commit - [#5208](#5208) `fix(clickhouse): restore the archive a Backup was taken as`, 3 commits - [#5230](#5230) `fix(backups): re-render the default backup objects when the S3 endpoint changes`, 8 commits - [#5133](#5133) `fix(securitygroups): project a membership-only group to a policy Cilium accepts`, 1 commit - [#5091](#5091) `fix(postgres): render the declared databases in a stable order`, 1 commit - [#5123](#5123) `fix(mariadb-operator): derive the cluster name from the platform domain`, 2 commits - [#5151](#5151) `fix(redis): run one Sentinel when Redis runs a single pod`, 5 commits - [#5124](#5124) `fix(lineage, kubernetes): resolve CAPK node VMs to their cluster`, 2 commits - [#5184](#5184) `fix(api): read pool usage under the quota key the controller counts`, 1 commit - [#5190](#5190) `fix(platform): remove the dead registries value and patch-containerd Secret`, 2 commits - [#5211](#5211) `fix(postgres): fail a backup job as soon as its cnpg Backup fails`, 1 commit - [#5155](#5155) `fix(postgres): refuse system databases and never drop them`, 1 commit - [#5207](#5207) `fix(mariadb): fail backup and restore jobs whose operator Job failed`, 5 commits - [#5024](#5024) `fix(kubernetes, monitoring): send tenant cluster metrics to the tenant's vminsert`, 1 commit ### Maintainer overrides - [#5083](#5083) `fix(workloadmonitor): stop doubling the bucket- prefix in bucket Workload names`: `backport/skip`, left out - [#4973](#4973) `fix(kubernetes-nodes): refuse a partial resources spec on every pool`: `backport/skip`, left out - [#5180](#5180) `fix(tests): keep two unit tests green after the release image restamp`: `backport/skip`, left out ### Skipped - Rule 1, already on `release-1.7`: 19 - Rule 2, the title type is not `fix` or `security`, or the scope is `deps`: 26 - Rule 3, a breaking change: 5 - Rule 4, touches platform migrations, `targetVersion` or a CRD, or makes a sizeable API change: 2 <details><summary>Rule 1, already on `release-1.7`</summary> - [#4815](#4815) `fix(vm-disk): name the importer error in the disk's WorkloadsReady message`, `its 2 commits picked, the last in 4323363` - [#4293](#4293) `fix(fluxcd): keep Helm uninstall working during flux-plunger recovery`, `its 1 commit picked, the last in 0210407` - [#4791](#4791) `fix(kubernetes): keep NFS publish from hanging on a dead server`, `its 1 commit picked, the last in fa88bd1` - [#4830](#4830) `fix(multus): drop the CPU limit and make resources configurable`, `its 2 commits picked, the last in 363c842` - [#4865](#4865) `fix(kafka): pass the container limits to the Kafka JVMs`, `its 3 commits picked, the last in fc38a54` - [#4866](#4866) `fix(linstor): mark volumes restored from a snapshot DRBD-initialized`, `its 1 commit picked, the last in f9550b7` - [#4819](#4819) `fix(seaweedfs): build SeaweedFS in-tree with a bounded S3 read-ahead`, `its 5 commits picked, the last in 7f4c346` - [#3427](#3427) `fix(backups): prepare standalone VMDisk restore targets and add VM backup/restore e2e round-trips`, `its 3 commits picked, the last in 4ac0cd5` - [#4889](#4889) `fix(monitoring): send alerts to every Alertmanager replica`, `its 1 commit picked, the last in 5033d53` - [#4929](#4929) `fix(backups): gate every default Strategy CR on its CRD`, `its 2 commits picked, the last in dee1eef` - [#4914](#4914) `fix(fluxcd): close the flux-plunger review follow-ups for the hook, drain and Warnings`, `its 7 commits picked, the last in a8e4916` - [#4939](#4939) `fix(cluster-api): resync capi-operator every minute`, `its 1 commit picked, the last in cc8d6a4` - [#5027](#5027) `fix(hack): keep the backup stack in the vminstance selective install`, `its 1 commit picked, the last in 8ba8386` - [#4955](#4955) `fix(vm-disk): let CDI adopt the PVC a Velero restore recreates`, `its 2 commits picked, the last in 691d7bc` - [#5071](#5071) `fix(marketplace): do not install a connected repository's components on connect`, `its 2 commits picked, the last in 25f8f11` - [#5086](#5086) `fix(tenant): stop the gateway release waiting on TenantGateway listeners and certificates`, `its 1 commit picked, the last in 18b2249` - [#5087](#5087) `fix(controller): carry healthCheckExprs and waitStrategy to existing releases`, `its 1 commit picked, the last in d2a452e` - [#5088](#5088) `fix(ingress-nginx): drop the admission-webhook policy a rebase brought back`, `its 1 commit picked, the last in cf00b41` - [#5142](#5142) `fix(release): keep release bodies within GitHub's size limit`, `its 1 commit picked, the last in c740515` </details> <details><summary>Rule 2, the title type is not `fix` or `security`, or the scope is `deps`</summary> - [#4845](#4845) `docs(governance): welcome new contributors and link LF sanctions guidance`, `type docs` - [#4881](#4881) `feat(bucket): add a ui flag to skip the s3manager web UI`, `type feat` - [#4899](#4899) `chore: link the FOSSA license scan for CLOMonitor`, `type chore` - [#4898](#4898) `feat(backups): back up Valkey with the Redis strategy`, `type feat` - [#4940](#4940) `docs(release): add changelog for v1.6.5`, `type docs` - [#4861](#4861) `feat(release): sign the release assets and attach a source SBOM`, `type feat` - [#3780](#3780) `docs: move the operator guides out of docs/ to where readers find them`, `type docs` - [#4936](#4936) `docs(release): check changelog claims against the code`, `type docs` - [#4959](#4959) `docs(release): point changelog check B at the Talos installer profile`, `type docs` - [#3777](#3777) `docs(agents): define where a new document belongs`, `type docs` - [#5025](#5025) `test(harbor): pin the resources of the nginx proxy and the portal`, `type test` - [#5039](#5039) `feat(keycloak): load additional CA files from Secret or ConfigMap keys`, `type feat` - [#4860](#4860) `feat(kubernetes-nodes): trust a factory CA in the Proxmox template script`, `type feat` - [#5090](#5090) `ci(e2e): end the Proxmox lane's SSH key with a newline`, `type ci` - [#4969](#4969) `test(kubernetes): pin RetryOnFailure on the Proxmox csi release`, `type test` - [#5000](#5000) `test(kubernetes): name the Proxmox fixture's node pool <cluster>-<pool>`, `type test` - [#5168](#5168) `docs(agents): steer agents away from Helm lookup`, `type docs` - [#4997](#4997) `feat(release): attach an SBOM to every release image`, `type feat` - [#3440](#3440) `ci(api-review-gate): publish the gate as a commit status`, `type ci` - [#5034](#5034) `docs: move the repo-only guides onto the website`, `type docs` - [#5215](#5215) `feat(monitoring-agents): export PVC labels from kube-state-metrics`, `type feat` - [#5217](#5217) `feat(monitoring-agents): keep application identity in container log streams`, `type feat` - [#5171](#5171) `feat(api): expose HelmRelease failure state on Application status`, `type feat` - [#5237](#5237) `feat(operator): let the platform source trust a private registry CA`, `type feat` - [#5106](#5106) `feat(release): make backports automatic with backport-batch`, `type feat` - [#4983](#4983) `feat(api): add the reservation evaluator for tenant quotas`, `type feat` </details> <details><summary>Rule 3, a breaking change</summary> - [#4790](#4790) `fix(platform)!: refuse a networking variant change on a live cluster`, `! in the title` - [#4796](#4796) `fix(vm-disk)!: refuse a golden-image clone CDI would leave unbound`, `! in the title` - [#5085](#5085) `fix(kubernetes, opensearch, foundationdb)!: restrict managed application images`, `! in the title` - [#4818](#4818) `fix(keycloak)!: make the KMS proxy keep its keys across restarts`, `! in the title` - [#5167](#5167) `fix(monitoring-agents)!: isolate merged log payload under log_processed`, `! in the title` </details> <details><summary>Rule 4, touches platform migrations, `targetVersion` or a CRD, or makes a sizeable API change</summary> - [#4941](#4941) `fix(backups): keep the backupstrategy-controller Helm release under the Secret size limit`, `a sizeable API change (breaking-change: [v1alpha1] spec.spec.template: field "metadata" was removed, and 5 more), a CRD` - [#5223](#5223) `fix(backups): report sizeBytes for MariaDB and CNPG backup artifacts`, `a CRD` </details> ```backport-batch {"line":"release-1.7","deferred":[]} ``` ### Screenshots See the original pull requests. ### Downstream repositories - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(backupstrategy-controller): the ClickHouse (Altinity) and RabbitMQ backup strategy Pods now run a dedicated `backup-http-client` image (alpine + curl + jq + ca-certificates, non-root, ~7 MB) instead of the 349 MB `platform-migrations` image. Air-gapped mirrors need the new `ghcr.io/cozystack/cozystack/backup-http-client` repository. fix(e2e): a chainsaw assert in the e2e suites keeps polling through a transient API error such as an etcd leader change, instead of failing the suite long before its timeout. fix(ci): use GITHUB_TOKEN to start the initial packages build when freezing a release line fix(etcd): the tenant etcd alert etcdHighNumberOfFailedGRPCRequests now counts only the gRPC codes that the etcd mixin and the platform copy of the alert count, so the Canceled watch streams of a Kubernetes API server that stops no longer fire it on every etcd member. fix(tcp-balancer): HAProxy now takes its fd and thread caps from the replica's memory and CPU limits instead of the node, so a t1.nano balancer is no longer OOMKilled on nodes with many CPUs. maxconn follows the memory limit at about 16 connections per MiB, 2035 on t1.nano, and the pods roll once on upgrade. fix(kubernetes): the Proxmox cloud-controller-manager now runs v0.16.1, and the Proxmox CCM and CSI driver images are pinned by digest. The v0.15.0 tag referenced before was re-pushed upstream to a build that panics on start, which left every new Proxmox worker with the uninitialized taint. fix(marketplace): a tapped repository is pinned to a digest instead of following a mutable tag, so moving the tag upstream no longer changes what runs in the cluster; connect accepts oci://...@sha256:... and an optional keyless cosign verify policy (anchored to an exact identity and preserved across re-connects), and with verify flags the CLI verifies the signature and pins the verified digest at tap time. fix(dashboard): tenant name in the tenant list links to its detail page now (where Delete is), and top level tenants get their Edit button back. fix(cilium): the IPAM migration hook no longer fails with "Argument list too long" on clusters with more than a few nodes, which blocked the cilium upgrade and every package that depends on networking. fix(kubernetes): the OIDC bootstrap Job no longer fails to start for a tenant cluster with about 240 or more oidc.users, the bindings now come from a ConfigMap the Job mounts. fix(clickhouse): a RestoreJob now restores the archive its Backup was taken as, not the newest one. New Backups record the archive name, Backups taken before this release are restored best effort from the newest archive stamped no later than their takenAt, with a warning in the restore pod log. A ClickHouse BackupJob created again under a name whose Backup still exists now fails instead of reporting that Backup as its result. fix(backups): the backup controller now re-renders the Velero BSL, the default backup strategies and the platform bucket's credentials Secret when their S3 endpoint is stale, so after the platform bucket's BucketAccess is re-granted for a new publishing.host, backups move to the new endpoint without forcing releases by hand. fix(securitygroups): a SecurityGroup with attachments and no rules no longer leaves an invalid CiliumNetworkPolicy behind; its backing policy is now accepted by Cilium and still allows or denies nothing. fix(postgres): the init script renders the declared databases in a stable order, so an upgrade that leaves the databases unchanged no longer rewrites the init-script Secret. fix(mariadb-operator): the MariaDB operator now uses the platform's networking.clusterDomain instead of a hard-coded cozy.local, so replication and User/Database/Grant reconciliation work on clusters with a non-default cluster domain. fix(redis): an instance with `replicas: 1` now runs one Sentinel instead of three, without a Sentinel PodDisruptionBudget; on upgrade, the PDB the operator created earlier is deleted. Instances with two replicas or more are unchanged. fix(lineage, kubernetes): the virt-launcher pods of Kubernetes worker nodes are now resolved to their Kubernetes application and marked as managed by Cozystack, so a SecurityGroup attached to a Kubernetes cluster now applies to its worker nodes, not only to its control plane. fix(api): tenant quota admission now counts the parent pool's usage of extended resources and hugepages, so a child tenant can no longer get a share the parent has already used. After upgrading, updates to existing tenants may be rejected if their quota for these resources exceeds the parent's remaining budget. This includes updates that leave the quota unchanged. fix(postgres): a Postgres BackupJob now fails as soon as its CloudNativePG Backup fails, with the recorded error, instead of after 30 minutes, and a BackupJob waiting on a CloudNativePG Backup reports a WaitingForCNPGBackup condition with the Backup's phase. fix(postgres): databases now refuses the PostgreSQL system databases postgres, template0 and template1, and the init job never runs DROP DATABASE on them. A system database a previous version took over is handed back to the superuser on the next upgrade (event trigger removed, ownership and public schema restored, its <db>_admin and <db>_readonly roles dropped). A release that declares one of them must remove the entry before upgrading. fix(mariadb): a MariaDB BackupJob or RestoreJob whose operator Job failed now ends Failed instead of Succeeded, and a failed backup no longer creates a Ready Backup without an archive. Backups created by failed runs before this fix are not removed; restoring from one can fail or select another dump from the shared prefix. fix(kubernetes, monitoring): the vmagent of tenant Kubernetes clusters now writes to the tenant's vminsert and labels series with the owning tenant instead of `cozy-monitoring`. Previously no tenant cluster metric reached the tenant's Monitoring. ```
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does The postgres init script lists the declared databases with `keys .Values.databases`, which returns them in Go's randomized map order. A release with two or more databases therefore renders a different `DEFINED_DBS` line on each upgrade, so the `<release>-init-script` Secret and the `checksum/config` annotation of the init Job change on upgrades that did not touch the databases. Sorting the keys makes the render a function of the values alone. Found while measuring a platform-wide values change on a production cluster (#5092): of 36 releases that ran Jobs again, 31 rendered a byte-identical manifest across the two Helm revisions and the other 5 were Postgres releases whose only difference was this ordering, e.g. `DEFINED_DBS="db_a db_b"` → `DEFINED_DBS="db_b db_a"`. This does not stop the init Job from running on every upgrade (it is a `post-upgrade` hook); the issue covers that part. Verification: the new `tests/init_script_deterministic_test.yaml` renders eight databases declared out of order and expects them sorted. Without the fix it failed 5 runs out of 5 with a different order each time; with it, `helm unittest packages/apps/postgres` passes (27 suites, 210 tests). ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(postgres): the init script renders the declared databases in a stable order, so an upgrade that leaves the databases unchanged no longer rewrites the init-script Secret. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **Bug Fixes** * Database initialization now lists configured databases in alphabetical order, providing consistent output across deployments. * **Tests** * Added coverage to verify the alphabetical ordering. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Backported-from: 0c48626
## What this PR does Backports to `release-1.7` from `main`, picked by the rules of the [backport policy](https://github.com/cozystack/cozystack/blob/main/docs/release/backports.md) in merge order. A merged pull request comes as its own commits, picked with `git cherry-pick -x` so each keeps its author, message and sign-offs, and a merge commit that carries the original's merge message with a `Backported-from:` trailer naming it; a squash-merged one comes as one pick. `backport-batch verify` checks that shape. A pick listed as changed differs from the commit it came from; `git range-diff <merge>^1..<merge>^2 <batch merge>^1..<batch merge>^2` shows how. A conflict or a failing check here is a [backport task](https://github.com/cozystack/cozystack/blob/main/docs/release/backport-tasks/README.md). - [#5284](#5284) `fix(seaweedfs): bump the COSI driver to v0.4.0 for the lost-grant fix`, 1 commit ### Maintainer overrides - [#5083](#5083) `fix(workloadmonitor): stop doubling the bucket- prefix in bucket Workload names`: `backport/skip`, left out - [#4973](#4973) `fix(kubernetes-nodes): refuse a partial resources spec on every pool`: `backport/skip`, left out - [#5180](#5180) `fix(tests): keep two unit tests green after the release image restamp`: `backport/skip`, left out ### Skipped - Rule 1, already on `release-1.7`: 46 - Rule 2, the title type is not `fix` or `security`, or the scope is `deps`: 31 - Rule 3, a breaking change: 5 - Rule 4, touches platform migrations, `targetVersion` or a CRD, or makes a sizeable API change: 2 <details><summary>Rule 1, already on `release-1.7`</summary> - [#4815](#4815) `fix(vm-disk): name the importer error in the disk's WorkloadsReady message`, `its 2 commits picked, the last in 4323363` - [#4756](#4756) `fix(backupstrategy-controller): run backup clients on a package-owned image`, `its 2 commits picked, the last in 4643366` - [#4293](#4293) `fix(fluxcd): keep Helm uninstall working during flux-plunger recovery`, `its 1 commit picked, the last in 0210407` - [#4791](#4791) `fix(kubernetes): keep NFS publish from hanging on a dead server`, `its 1 commit picked, the last in fa88bd1` - [#4830](#4830) `fix(multus): drop the CPU limit and make resources configurable`, `its 2 commits picked, the last in 363c842` - [#4865](#4865) `fix(kafka): pass the container limits to the Kafka JVMs`, `its 3 commits picked, the last in fc38a54` - [#4866](#4866) `fix(linstor): mark volumes restored from a snapshot DRBD-initialized`, `its 1 commit picked, the last in f9550b7` - [#4867](#4867) `fix(e2e): keep chainsaw asserts polling through transient API errors`, `its 1 commit picked, the last in df75adc` - [#4819](#4819) `fix(seaweedfs): build SeaweedFS in-tree with a bounded S3 read-ahead`, `its 5 commits picked, the last in 7f4c346` - [#3427](#3427) `fix(backups): prepare standalone VMDisk restore targets and add VM backup/restore e2e round-trips`, `its 3 commits picked, the last in 4ac0cd5` - [#4889](#4889) `fix(monitoring): send alerts to every Alertmanager replica`, `its 1 commit picked, the last in 5033d53` - [#4929](#4929) `fix(backups): gate every default Strategy CR on its CRD`, `its 2 commits picked, the last in dee1eef` - [#4914](#4914) `fix(fluxcd): close the flux-plunger review follow-ups for the hook, drain and Warnings`, `its 7 commits picked, the last in a8e4916` - [#4888](#4888) `fix(ci): dispatch frozen-line builds with GITHUB_TOKEN`, `its 1 commit picked, the last in bf81e2d` - [#4939](#4939) `fix(cluster-api): resync capi-operator every minute`, `its 1 commit picked, the last in cc8d6a4` - [#5027](#5027) `fix(hack): keep the backup stack in the vminstance selective install`, `its 1 commit picked, the last in 8ba8386` - [#4955](#4955) `fix(vm-disk): let CDI adopt the PVC a Velero restore recreates`, `its 2 commits picked, the last in 691d7bc` - [#4985](#4985) `fix(etcd): stop counting canceled watch streams as failed gRPC requests`, `its 1 commit picked, the last in 38e4089` - [#5005](#5005) `fix(tcp-balancer): cap HAProxy fds and threads by the pod's own limits`, `its 1 commit picked, the last in 7a3d0b0` - [#5071](#5071) `fix(marketplace): do not install a connected repository's components on connect`, `its 2 commits picked, the last in 25f8f11` - [#5086](#5086) `fix(tenant): stop the gateway release waiting on TenantGateway listeners and certificates`, `its 1 commit picked, the last in 18b2249` - [#5087](#5087) `fix(controller): carry healthCheckExprs and waitStrategy to existing releases`, `its 1 commit picked, the last in d2a452e` - [#5088](#5088) `fix(ingress-nginx): drop the admission-webhook policy a rebase brought back`, `its 1 commit picked, the last in cf00b41` - [#5142](#5142) `fix(release): keep release bodies within GitHub's size limit`, `its 1 commit picked, the last in c740515` - [#4917](#4917) `fix(kubernetes): run Proxmox CCM v0.16.1 and pin the Proxmox images by digest`, `its 2 commits picked, the last in 8ff6dca` - [#4974](#4974) `fix(kubernetes-nodes): plan autoscaler capacity on the cores the VM actually gets`, `its 1 commit picked, the last in d68a163` - [#4971](#4971) `fix(e2e): keep the Proxmox tenant kubeconfig per run and remove it`, `its 2 commits picked, the last in 77227c5` - [#4970](#4970) `fix(e2e): read the KubernetesNodes pool release in the HelmRelease guard`, `its 3 commits picked, the last in 6b17a76` - [#4972](#4972) `fix(e2e): validate the leak allowlist against every suite layout`, `its 3 commits picked, the last in 7ec550b` - [#5082](#5082) `fix(marketplace): pin a tapped repository to a digest instead of a mutable tag`, `its 15 commits picked, the last in 9cff86f` - [#4004](#4004) `fix(dashboard): link a tenant row to its detail page`, `its 2 commits picked, the last in bd5e0b5` - [#5212](#5212) `fix(cilium): pass the CiliumNode list to jq on stdin in the IPAM hook`, `its 1 commit picked, the last in b28a2e2` - [#5213](#5213) `fix(kubernetes): move the OIDC users out of the bootstrap Job's script`, `its 1 commit picked, the last in 02dd8e5` - [#5208](#5208) `fix(clickhouse): restore the archive a Backup was taken as`, `its 3 commits picked, the last in 9ded5c5` - [#5230](#5230) `fix(backups): re-render the default backup objects when the S3 endpoint changes`, `its 8 commits picked, the last in 490eab5` - [#5133](#5133) `fix(securitygroups): project a membership-only group to a policy Cilium accepts`, `its 1 commit picked, the last in b7c50a5` - [#5091](#5091) `fix(postgres): render the declared databases in a stable order`, `its 1 commit picked, the last in b658dff` - [#5123](#5123) `fix(mariadb-operator): derive the cluster name from the platform domain`, `its 2 commits picked, the last in 17702dd` - [#5151](#5151) `fix(redis): run one Sentinel when Redis runs a single pod`, `its 5 commits picked, the last in 5524a41` - [#5124](#5124) `fix(lineage, kubernetes): resolve CAPK node VMs to their cluster`, `its 2 commits picked, the last in c908560` - [#5184](#5184) `fix(api): read pool usage under the quota key the controller counts`, `its 1 commit picked, the last in 18c17af` - [#5190](#5190) `fix(platform): remove the dead registries value and patch-containerd Secret`, `its 2 commits picked, the last in 617f889` - [#5211](#5211) `fix(postgres): fail a backup job as soon as its cnpg Backup fails`, `its 1 commit picked, the last in 6645e21` - [#5155](#5155) `fix(postgres): refuse system databases and never drop them`, `its 1 commit picked, the last in 6f1bb80` - [#5207](#5207) `fix(mariadb): fail backup and restore jobs whose operator Job failed`, `its 5 commits picked, the last in ef3de55` - [#5024](#5024) `fix(kubernetes, monitoring): send tenant cluster metrics to the tenant's vminsert`, `its 1 commit picked, the last in 423a75b` </details> <details><summary>Rule 2, the title type is not `fix` or `security`, or the scope is `deps`</summary> - [#4845](#4845) `docs(governance): welcome new contributors and link LF sanctions guidance`, `type docs` - [#4881](#4881) `feat(bucket): add a ui flag to skip the s3manager web UI`, `type feat` - [#4899](#4899) `chore: link the FOSSA license scan for CLOMonitor`, `type chore` - [#4898](#4898) `feat(backups): back up Valkey with the Redis strategy`, `type feat` - [#4940](#4940) `docs(release): add changelog for v1.6.5`, `type docs` - [#4861](#4861) `feat(release): sign the release assets and attach a source SBOM`, `type feat` - [#3780](#3780) `docs: move the operator guides out of docs/ to where readers find them`, `type docs` - [#4936](#4936) `docs(release): check changelog claims against the code`, `type docs` - [#4959](#4959) `docs(release): point changelog check B at the Talos installer profile`, `type docs` - [#3777](#3777) `docs(agents): define where a new document belongs`, `type docs` - [#5025](#5025) `test(harbor): pin the resources of the nginx proxy and the portal`, `type test` - [#5039](#5039) `feat(keycloak): load additional CA files from Secret or ConfigMap keys`, `type feat` - [#4860](#4860) `feat(kubernetes-nodes): trust a factory CA in the Proxmox template script`, `type feat` - [#5090](#5090) `ci(e2e): end the Proxmox lane's SSH key with a newline`, `type ci` - [#4969](#4969) `test(kubernetes): pin RetryOnFailure on the Proxmox csi release`, `type test` - [#5000](#5000) `test(kubernetes): name the Proxmox fixture's node pool <cluster>-<pool>`, `type test` - [#5168](#5168) `docs(agents): steer agents away from Helm lookup`, `type docs` - [#4997](#4997) `feat(release): attach an SBOM to every release image`, `type feat` - [#3440](#3440) `ci(api-review-gate): publish the gate as a commit status`, `type ci` - [#5034](#5034) `docs: move the repo-only guides onto the website`, `type docs` - [#5215](#5215) `feat(monitoring-agents): export PVC labels from kube-state-metrics`, `type feat` - [#5217](#5217) `feat(monitoring-agents): keep application identity in container log streams`, `type feat` - [#5171](#5171) `feat(api): expose HelmRelease failure state on Application status`, `type feat` - [#5237](#5237) `feat(operator): let the platform source trust a private registry CA`, `type feat` - [#5106](#5106) `feat(release): make backports automatic with backport-batch`, `type feat` - [#4983](#4983) `feat(api): add the reservation evaluator for tenant quotas`, `type feat` - [#5264](#5264) `feat(release): keep a line's backport record and reports across batch pull requests`, `type feat` - [#5260](#5260) `ci(e2e): skip final cleanup on disposable PR runners`, `type ci` - [#5046](#5046) `feat(backups): record the RDB snapshot location and size for Redis backups`, `type feat` - [#5214](#5214) `feat(observability-proxy): serve tenant namespace logs through an authorizing proxy`, `type feat` - [#5282](#5282) `test(release): isolate report deduplication from workflow attempts`, `type test` </details> <details><summary>Rule 3, a breaking change</summary> - [#4790](#4790) `fix(platform)!: refuse a networking variant change on a live cluster`, `! in the title` - [#4796](#4796) `fix(vm-disk)!: refuse a golden-image clone CDI would leave unbound`, `! in the title` - [#5085](#5085) `fix(kubernetes, opensearch, foundationdb)!: restrict managed application images`, `! in the title` - [#4818](#4818) `fix(keycloak)!: make the KMS proxy keep its keys across restarts`, `! in the title` - [#5167](#5167) `fix(monitoring-agents)!: isolate merged log payload under log_processed`, `! in the title` </details> <details><summary>Rule 4, touches platform migrations, `targetVersion` or a CRD, or makes a sizeable API change</summary> - [#4941](#4941) `fix(backups): keep the backupstrategy-controller Helm release under the Secret size limit`, `a sizeable API change (breaking-change: [v1alpha1] spec.spec.template: field "metadata" was removed, and 5 more), a CRD` - [#5223](#5223) `fix(backups): report sizeBytes for MariaDB and CNPG backup artifacts`, `a CRD` </details> ```backport-batch {"version":2,"line":"release-1.7","built":{"main":"76a34d3e53a14c39d8df63eec6271c2eaae5ffea","line":"3cd9d4c220b519972677c7ff756ae6c4db07ef4a","newer":"","policy":"76a34d3e53a14c39d8df63eec6271c2eaae5ffea","batch":"e27d6630a0008381f0bc105a87d117dfcc29680f","tool":"76a34d3e53a14c39d8df63eec6271c2eaae5ffea","run":"https://github.com/cozystack/cozystack/actions/runs/43209950053","at":"2026-11-12T11:33:16Z","options":{"noWorkflowFiles":true,"mode":"new"}},"report":"9ca0285a0bb30a09736f6a430878b3ce0d32a2811639861e80daa549f208d7b3","closing":null,"deferred":[],"dropped":[],"stale":[]} ``` ```backport-batch-report {"format":1,"id":"9ca0285a0bb30a09736f6a430878b3ce0d32a2811639861e80daa549f208d7b3","line":"release-1.7","built":{"main":"76a34d3e53a14c39d8df63eec6271c2eaae5ffea","line":"3cd9d4c220b519972677c7ff756ae6c4db07ef4a","newer":"","policy":"76a34d3e53a14c39d8df63eec6271c2eaae5ffea","batch":"e27d6630a0008381f0bc105a87d117dfcc29680f","tool":"76a34d3e53a14c39d8df63eec6271c2eaae5ffea","run":"https://github.com/cozystack/cozystack/actions/runs/43209950053","at":"2026-11-12T11:33:16Z","options":{"noWorkflowFiles":true,"mode":"new"}},"overrides":{"backport/include":[],"backport/skip":[4973,5083,5180]},"primary":{"picked":[5284],"missing":[],"deferred":[],"parked":[],"skipped":{"on-line":[3427,4004,4293,4756,4791,4815,4819,4830,4865,4866,4867,4888,4889,4914,4917,4929,4939,4955,4970,4971,4972,4974,4985,5005,5024,5027,5071,5082,5086,5087,5088,5091,5123,5124,5133,5142,5151,5155,5184,5190,5207,5208,5211,5212,5213,5230],"before-start":[],"type":[3440,3777,3780,4845,4860,4861,4881,4898,4899,4936,4940,4959,4969,4983,4997,5000,5025,5034,5039,5046,5090,5106,5168,5171,5214,5215,5217,5237,5260,5264,5282],"breaking":[4790,4796,4818,5085,5167],"touches":[4941,5223],"newer":[],"skip-label":[4973,5083,5180],"dropped":[]}},"overlays":{"dropped":[],"stale":[],"changed":[]},"diag":{},"other":[]} ``` ### Screenshots See the original pull requests. ### Downstream repositories - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(seaweedfs): bucket users granted together could get S3 keys that never work (403 InvalidAccessKeyId). COSI driver is bumped to v0.4.0, which writes each grant to per-identity IAM file. ```
What this PR does
The postgres init script lists the declared databases with
keys .Values.databases, which returns them in Go's randomized map order. A release with two or more databases therefore renders a differentDEFINED_DBSline on each upgrade, so the<release>-init-scriptSecret and thechecksum/configannotation of the init Job change on upgrades that did not touch the databases. Sorting the keys makes the render a function of the values alone.Found while measuring a platform-wide values change on a production cluster (#5092): of 36 releases that ran Jobs again, 31 rendered a byte-identical manifest across the two Helm revisions and the other 5 were Postgres releases whose only difference was this ordering, e.g.
DEFINED_DBS="db_a db_b"→DEFINED_DBS="db_b db_a".This does not stop the init Job from running on every upgrade (it is a
post-upgradehook); the issue covers that part.Verification: the new
tests/init_script_deterministic_test.yamlrenders eight databases declared out of order and expects them sorted. Without the fix it failed 5 runs out of 5 with a different order each time; with it,helm unittest packages/apps/postgrespasses (27 suites, 210 tests).Screenshots
Downstream repositories
Release note
Summary by CodeRabbit