Skip to content

Commit 991fc73

Browse files
committed
docs(db-autoscaler): document the kube-state-metrics monitoring requirement
Live testing surfaced an operational gotcha: the Postgres/MariaDB adapters recover the read-replica role by joining engine metrics against kube_pod_labels, which is cluster-scoped (scraped only by the root monitoring). A target whose namespace resolves to a tenant-local monitoring stack (monitoring: true) has the engine metrics but not kube_pod_labels, so the role join is empty and the DHA fail-safe-freezes with MetricUnavailable instead of scaling. Document the requirement and the common working layout (monitoring: false, pointing at root) in the package README. Signed-off-by: Alexey Artamonov <[email protected]>
1 parent b1310f8 commit 991fc73

1 file changed

Lines changed: 4 additions & 0 deletions

File tree

‎packages/system/db-autoscaler/README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,10 @@ bundles:
3535
3636
It declares a `dependsOn` on the monitoring stack (VictoriaMetrics / `WorkloadMonitor`), which the decision loop requires, and on cert-manager, which issues the webhook serving certificate.
3737

38+
## Monitoring requirements
39+
40+
The autoscaler resolves the target's vmselect from the `namespace.cozystack.io/monitoring` label on the target's namespace, then runs its PromQL there. For the Postgres and MariaDB adapters that vmselect must serve **both** the engine's own per-pod metrics (e.g. `cnpg_backends_total`) **and** `kube_pod_labels` from kube-state-metrics, because the read-replica role is not carried on the engine metrics and is recovered by joining against `kube_pod_labels` on `label_cnpg_io_instance_role`. kube-state-metrics is cluster-scoped and is scraped only by the root/cluster monitoring, so this holds for the common tenant layout — a tenant with `monitoring: false`, whose namespace label points at the parent (root) monitoring where both series coexist (verified on a live cluster). A tenant that runs its **own** isolated stack (`monitoring: true`) scrapes the engine metrics but not cluster-scoped `kube_pod_labels`; the role join then returns no series, the metric reads as unavailable, and the DHA fail-safe-freezes (`AbleToScale=False`, reason `MetricUnavailable`) rather than scaling blind. Point such targets at a monitoring stack that also scrapes kube-state-metrics before enabling autoscaling.
41+
3842
## Ownership enforcement
3943

4044
While a DHA is active the autoscaler is the single owner of the target's `replicas` value: it stamps the marker annotation `autoscaling.cozystack.io/managed-by: <dha-name>` and writes via the `db-autoscaler` field manager.

0 commit comments

Comments
 (0)