Skip to content

Commit 03a2edf

Browse files
committed
Merge branch 'main' into feat/observability-proxy-app-scope
Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <[email protected]>
2 parents a4ea942 + 4e6ce46 commit 03a2edf

114 files changed

Lines changed: 4820 additions & 4324 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎api/apps/v1alpha1/vminstance/types.go‎

Lines changed: 3 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎api/apps/v1alpha1/vminstance/zz_generated.deepcopy.go‎

Lines changed: 8 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/changelogs/v1.7.1.md‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
<!--
2+
https://github.com/cozystack/cozystack/releases/tag/v1.7.1
3+
-->
4+
5+
# Cozystack v1.7.1
6+
7+
Cozystack v1.7.1 is a patch release on the v1.7 line. It fixes the COSI driver bug that could leave freshly granted S3 keys rejected for good, makes marketplace taps digest-pinned, makes MariaDB, PostgreSQL and ClickHouse backups report their real outcome, and repairs several tenant-facing issues: tenant cluster metrics, SecurityGroups on Kubernetes workers, Redis with a single replica, the Proxmox cloud-controller-manager and the tcp-balancer. Read the Migration and Upgrades section before upgrading.
8+
9+
## Fixes
10+
11+
* **SeaweedFS COSI driver v0.4.0**: bucket users granted together could get S3 keys that never work (`403 InvalidAccessKeyId`) because the v0.3.1 driver rewrote a legacy IAM file the S3 gateways were migrating. The driver is bumped to v0.4.0, which writes each grant to a per-identity IAM file. Keys already lost on running clusters stay broken: remove the user from the Bucket, wait until its BucketAccess is gone, and add it back ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #5284, backport #5290).
12+
* **Marketplace taps are pinned to a digest**: a tapped repository no longer follows a mutable tag, so moving the tag upstream cannot change what runs in the cluster. `tap connect` accepts `oci://...@sha256:...` references and an optional keyless cosign verify policy, anchored to an exact identity and preserved across re-connects; with the verify flags the CLI verifies the signature and pins the verified digest at tap time. Community index entries can record a digest too ([**@IvanHunters**](https://github.com/IvanHunters) in #5082, backport #5246).
13+
* **Tenant cluster metrics reach the tenant's Monitoring**: the vmagent of tenant Kubernetes clusters now writes to the tenant's vminsert and labels series with the owning tenant instead of `cozy-monitoring`; before, no tenant cluster metric reached the tenant's Monitoring ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #5024, backport #5246).
14+
* **SecurityGroups apply to Kubernetes worker nodes**: the virt-launcher pods of Kubernetes worker nodes are now resolved to their Kubernetes application and marked as managed by Cozystack, so a SecurityGroup attached to a Kubernetes cluster applies to its workers, not only to its control plane ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #5124, backport #5246).
15+
* **Membership-only SecurityGroups**: a SecurityGroup with attachments and no rules no longer leaves an invalid CiliumNetworkPolicy behind; its backing policy is accepted by Cilium and still allows or denies nothing ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #5133, backport #5246).
16+
* **Proxmox CCM and CSI images**: the Proxmox cloud-controller-manager runs v0.16.1 and the Proxmox CCM and CSI driver images are pinned by digest. The v0.15.0 tag was re-pushed upstream to a build that panics on start, which left every new Proxmox worker with the uninitialized taint ([**@themoriarti**](https://github.com/themoriarti) in #4917, backport #5246).
17+
* **Autoscaler capacity of KubernetesNodes pools**: on the kubevirt substrate the cluster-autoscaler capacity annotation is planned on the cores the VM actually gets, so a leading-zero `resources.cpu` such as `"010"` (decoded as YAML, eight cores) no longer yields a wrong capacity. No pool rolls ([**@themoriarti**](https://github.com/themoriarti) in #4974, backport #5246).
18+
* **Tenant quota admission counts extended resources**: admission now reads the parent pool's usage of extended resources and hugepages under the `requests.*` key the controller counts, so a child tenant can no longer get a share the parent has already used ([**@lexfrei**](https://github.com/lexfrei) in #5184, backport #5246).
19+
* **Postgres backup jobs fail as soon as the CloudNativePG Backup fails**: a Postgres `BackupJob` now fails with the recorded error instead of after 30 minutes, and reports a `WaitingForCNPGBackup` condition with the Backup's phase while it waits ([**@scooby87**](https://github.com/scooby87) in #5211, backport #5246).
20+
* **MariaDB backups and restores report real failures**: a `BackupJob` or `RestoreJob` whose operator Job failed now ends Failed instead of Succeeded, and a failed backup no longer creates a Ready Backup without an archive ([**@scooby87**](https://github.com/scooby87) in #5207, backport #5246).
21+
* **ClickHouse restores use the right archive**: a `RestoreJob` restores the archive its Backup was taken as, not the newest one. New Backups record the archive name; Backups taken before this release are restored best effort from the newest archive stamped no later than their `takenAt`, with a warning in the restore pod log. A ClickHouse `BackupJob` created under a name whose Backup still exists now fails instead of reporting that Backup as its result ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #5208, backport #5246).
22+
* **Backup S3 endpoint changes are followed**: the backup controller re-renders the Velero BSL, the default backup strategies and the platform bucket's credentials Secret when their S3 endpoint is stale, so after the platform bucket's BucketAccess is re-granted for a new `publishing.host`, backups move to the new endpoint without forcing releases by hand ([**@scooby87**](https://github.com/scooby87) in #5230, backport #5246).
23+
* **Backup clients run on a dedicated image**: the ClickHouse (Altinity) and RabbitMQ backup strategy Pods now run a small `backup-http-client` image instead of the 349 MB `platform-migrations` image, so a platform migration change no longer re-digests the backup clients. Air-gapped mirrors need the new `ghcr.io/cozystack/cozystack/backup-http-client` repository ([**@androndo**](https://github.com/androndo) in #4756, backport #5246).
24+
* **PostgreSQL system databases**: `databases` refuses `postgres`, `template0` and `template1`, and the init job never runs `DROP DATABASE` on them, which used to fail the init job on every upgrade and leave the HelmRelease in `UpgradeFailed` ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #5155, backport #5246).
25+
* **PostgreSQL init script is stable**: the declared databases render in a stable order, so an upgrade that leaves the databases unchanged no longer rewrites the init-script Secret ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #5091, backport #5246).
26+
* **Redis with a single replica**: an instance with `replicas: 1` runs one Sentinel instead of three, without a Sentinel PodDisruptionBudget, so node drains are no longer blocked by it; instances with two replicas or more are unchanged ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #5151, backport #5246).
27+
* **MariaDB operator on a custom cluster domain**: the operator uses the platform's `networking.clusterDomain` instead of a hard-coded `cozy.local`, so replication and User/Database/Grant reconciliation work on clusters with a non-default domain ([**@mattia-eleuteri**](https://github.com/mattia-eleuteri) in #5123, backport #5246).
28+
* **tcp-balancer no longer OOMs**: HAProxy takes its fd and thread caps from the replica's own memory and CPU limits instead of the node, so a `t1.nano` balancer is no longer OOMKilled on nodes with many CPUs ([**@yankawai**](https://github.com/yankawai) in #5005, backport #5246).
29+
* **etcd alert noise**: `etcdHighNumberOfFailedGRPCRequests` counts only the gRPC codes the etcd mixin counts, so the Canceled watch streams of a stopping Kubernetes API server no longer fire it on every member ([**@yankawai**](https://github.com/yankawai) in #4985, backport #5246).
30+
* **Cilium IPAM hook on larger clusters**: the hook passes the CiliumNode list to `jq` on stdin and no longer fails with "Argument list too long" on clusters with more than a few nodes, which blocked the cilium upgrade and every package depending on networking ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #5212, backport #5246).
31+
* **OIDC bootstrap Job with many users**: the Job no longer fails to start for a tenant cluster with about 240 or more `oidc.users`; the bindings now come from a ConfigMap the Job mounts ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #5213, backport #5246).
32+
* **Dashboard tenant list links to the tenant page**: the tenant name in the tenant list opens its detail page, where Delete lives, and top-level tenants get their Edit button back ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4004, backport #5246).
33+
* **Dead `registries` platform value removed**: the platform chart no longer has the `registries` value or the `patch-containerd` Secret, which nothing read. Mirrors for tenant workers are the per-pool `talos.registryMirrors` of `KubernetesNodes` ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #5190, backport #5246).
34+
35+
## Security
36+
37+
* **Marketplace digest pinning and signature verification**: see the marketplace entry under Fixes (#5082).
38+
39+
## Development, Testing, and CI/CD
40+
41+
* **Chainsaw asserts survive transient API errors**: an e2e assert keeps polling through an error such as an etcd leader change instead of failing the suite early ([**@lexfrei**](https://github.com/lexfrei) in #4867, backport #5246).
42+
* **Proxmox e2e hygiene**: the Proxmox tenant kubeconfig is kept per run and removed afterwards, the HelmRelease guard reads the `KubernetesNodes` pool release, and the leak allowlist is validated against every suite layout ([**@themoriarti**](https://github.com/themoriarti) in #4971, #4970 and #4972, backports #5246).
43+
* **Frozen-line builds start with `GITHUB_TOKEN`**: the initial packages build dispatched when a release line is frozen now runs ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in #4888, backport #5246).
44+
* **Unit tests after the release image restamp**: two unit tests that read fields the release build rewrites stay green on `release-1.7`; this is a release-branch-only fix ([**@themoriarti**](https://github.com/themoriarti) in #5179).
45+
46+
## Documentation
47+
48+
* **[website] Documentation**: the v1.7 docs gain the guides that lived only in the cozystack repository, the pending v1.7 docs PRs, a DRBD note for the replicated storage class, a refreshed Chainsaw e2e guide, restored legacy URLs and a grouping of older doc versions ([**@myasnikovdaniil**](https://github.com/myasnikovdaniil) in cozystack/website#763, cozystack/website#782, cozystack/website#675, cozystack/website#611, cozystack/website#775 and cozystack/website#767).
49+
* **[website] Blog and demo**: a full-stack observability post for v1.7, the Cozystack 1.7 announcement, a three-board Raspberry Pi 5 post, and a collapsible navbar with a closable hint on the live demo page ([**@tym83**](https://github.com/tym83) in cozystack/website#790, cozystack/website#801 and cozystack/website#802; [**@kvaps**](https://github.com/kvaps) in cozystack/website#789).
50+
* **[website] Pages deploy**: the deploy workflow drops the Dart Sass install ([**@kvaps**](https://github.com/kvaps) in cozystack/website#798).
51+
* **[ansible-cozystack] Installer v1.7.0**: tracks the Cozystack installer image at v1.7.0, released in ansible-cozystack v1.7.0 ([**@app/renovate**](https://github.com/apps/renovate) in cozystack/ansible-cozystack#89).
52+
53+
## Migration and Upgrades
54+
55+
Platform migrations do not change in this release: `migrations.targetVersion` stays at 61.
56+
57+
* **Remove system databases from PostgreSQL `databases` before upgrading (#5155).** A release that declares `postgres`, `template0` or `template1` under `databases` is refused at render; remove the entry first. A system database a previous version took over is handed back to the superuser on the next upgrade: the event trigger is removed, ownership and the `public` schema are restored, and its `<db>_admin` and `<db>_readonly` roles are dropped.
58+
* **Tenant quota updates can be rejected after upgrading (#5184).** Admission now counts the parent's usage of extended resources and hugepages, so an update to an existing tenant can be rejected when its quota for these resources exceeds the parent's remaining budget, including an update that leaves the quota unchanged.
59+
* **Air-gapped mirrors need `backup-http-client` (#4756).** Mirror `ghcr.io/cozystack/cozystack/backup-http-client` before upgrading, or ClickHouse and RabbitMQ backup Pods cannot pull their image.
60+
* **tcp-balancer pods roll once (#5005).** The new HAProxy `global` section changes the rendered config; `maxconn` follows the memory limit at about 16 connections per MiB (2035 on `t1.nano`).
61+
* **Redis scaled to one replica loses its Sentinel PDB (#5151).** A post-upgrade hook deletes the Sentinel PodDisruptionBudget the operator created earlier for an instance with `replicas: 1`; scaling an instance between 1 and 2 replicas scales its Sentinel Deployment between 1 and 3.
62+
* **Backups created by failed MariaDB runs are not cleaned up (#5207).** Backups made by failed runs before this fix are not removed; restoring from one can fail or select another dump from the shared prefix.
63+
* **Keys lost to the COSI grant bug stay lost (#5284).** Re-add the affected user to the Bucket as described in the Fixes entry.
64+
* **`registries` in the platform Package is ignored (#5190).** A Package that still sets it keeps installing; the `patch-containerd` Secret in `cozy-system` is deleted by the release.
65+
* **SecurityGroups now reach Kubernetes workers (#5124).** A SecurityGroup attached to a `Kubernetes` application starts applying to its worker nodes after the upgrade, so review attached groups before upgrading.
66+
67+
## Contributors
68+
69+
We'd like to thank all contributors who made this release possible:
70+
71+
* [**@androndo**](https://github.com/androndo)
72+
* [**@IvanHunters**](https://github.com/IvanHunters)
73+
* [**@kvaps**](https://github.com/kvaps)
74+
* [**@lexfrei**](https://github.com/lexfrei)
75+
* [**@mattia-eleuteri**](https://github.com/mattia-eleuteri)
76+
* [**@myasnikovdaniil**](https://github.com/myasnikovdaniil)
77+
* [**@scooby87**](https://github.com/scooby87)
78+
* [**@themoriarti**](https://github.com/themoriarti)
79+
* [**@tym83**](https://github.com/tym83)
80+
* [**@yankawai**](https://github.com/yankawai)
81+
82+
**Full Changelog**: https://github.com/cozystack/cozystack/compare/v1.7.0...v1.7.1

‎hack/alert-severity-contract.bats‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,10 @@ load test_helper
99
# scope: they are re-vendored by `make update`, so a bad severity there is
1010
# fixed upstream or in patches/, not by editing the label.
1111
ALERT_SEVERITY_ALLOWED=" security critical major minor warning indeterminate informational normal ok cleared debug trace unknown "
12+
# The alerts that exist only to be dropped, the one place `none` is right.
13+
# InfoInhibitor is the source side of an Alertmanager inhibit rule, which the
14+
# platform config does not have, so delivering it at any severity is noise.
15+
ALERT_SEVERITY_BLACKHOLED=" InfoInhibitor "
1216

1317
helm_actions_stripped() {
1418
awk '
@@ -104,6 +108,14 @@ alert_severity_contract() {
104108
break
105109
fi
106110
done
111+
if [ "$severity" = "none" ]; then
112+
for blackholed in $ALERT_SEVERITY_BLACKHOLED; do
113+
if [ "$blackholed" = "$alert" ]; then
114+
accepted=1
115+
break
116+
fi
117+
done
118+
fi
107119
if [ "$accepted" -eq 0 ]; then
108120
echo "$file: $alert -> '$severity'" >&2
109121
bad=$((bad + 1))
@@ -128,6 +140,83 @@ alert_severity_contract() {
128140
alert_severity_contract
129141
}
130142

143+
@test "InfoInhibitor is labelled for the blackhole route" {
144+
rows=$(mktemp) || return 1
145+
if ! alert_severity_rows > "$rows"; then
146+
rm -f "$rows"
147+
return 1
148+
fi
149+
tab=$(printf '\t')
150+
found=$(grep -c "${tab}InfoInhibitor${tab}" "$rows" || true)
151+
delivered=$(grep "${tab}InfoInhibitor${tab}" "$rows" | grep -vc "${tab}none\$" || true)
152+
rm -f "$rows"
153+
[ "$found" -gt 0 ]
154+
[ "$delivered" -eq 0 ]
155+
}
156+
157+
@test "severity none passes only on an alert that exists to be dropped" {
158+
fixture=$(mktemp -d)
159+
mkdir -p "$fixture/packages/system/example/alerts"
160+
printf '%s\n' \
161+
'apiVersion: operator.victoriametrics.com/v1beta1' \
162+
'kind: VMRule' \
163+
'spec:' \
164+
' groups:' \
165+
' - name: general' \
166+
' rules:' \
167+
' - alert: InfoInhibitor' \
168+
' labels:' \
169+
' severity: none' \
170+
> "$fixture/packages/system/example/alerts/00-inhibitor.yaml"
171+
git -C "$fixture" init -q
172+
git -C "$fixture" add packages/system/example/alerts/00-inhibitor.yaml
173+
174+
inhibitor_succeeded=0
175+
ALERT_SEVERITY_ROOT="$fixture" alert_severity_contract >/dev/null 2>&1 || inhibitor_succeeded=$?
176+
177+
printf '%s\n' \
178+
'apiVersion: operator.victoriametrics.com/v1beta1' \
179+
'kind: VMRule' \
180+
'spec:' \
181+
' groups:' \
182+
' - name: silent' \
183+
' rules:' \
184+
' - alert: SilentAlert' \
185+
' labels:' \
186+
' severity: none' \
187+
> "$fixture/packages/system/example/alerts/01-silent.yaml"
188+
git -C "$fixture" add packages/system/example/alerts/01-silent.yaml
189+
190+
silent_succeeded=0
191+
ALERT_SEVERITY_ROOT="$fixture" alert_severity_contract >/dev/null 2>&1 || silent_succeeded=$?
192+
rm -rf "$fixture"
193+
[ "$inhibitor_succeeded" -eq 0 ]
194+
[ "$silent_succeeded" -ne 0 ]
195+
}
196+
197+
@test "an alert that exists to be dropped still fails on a refused severity" {
198+
fixture=$(mktemp -d)
199+
mkdir -p "$fixture/packages/system/example/alerts"
200+
printf '%s\n' \
201+
'apiVersion: operator.victoriametrics.com/v1beta1' \
202+
'kind: VMRule' \
203+
'spec:' \
204+
' groups:' \
205+
' - name: general' \
206+
' rules:' \
207+
' - alert: InfoInhibitor' \
208+
' labels:' \
209+
' severity: info' \
210+
> "$fixture/packages/system/example/alerts/00-inhibitor.yaml"
211+
git -C "$fixture" init -q
212+
git -C "$fixture" add packages/system/example/alerts/00-inhibitor.yaml
213+
214+
contract_succeeded=0
215+
ALERT_SEVERITY_ROOT="$fixture" alert_severity_contract >/dev/null 2>&1 || contract_succeeded=$?
216+
rm -rf "$fixture"
217+
[ "$contract_succeeded" -ne 0 ]
218+
}
219+
131220
@test "a parser failure cannot leave the contract green" {
132221
fixture=$(mktemp -d)
133222
mkdir -p "$fixture/packages/system/example/alerts"

0 commit comments

Comments
 (0)