You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 324815a
Browse filesBrowse the repository at this point in the historyBrowse files
docs(dha): correct the stock-HPA motivation (topology, not victim selection)
The motivation and the "Stock HPA + KEDA" alternative claimed HPA "would
happily remove a synchronous standby." That misattributes the mechanism:
HPA only writes a replica count (for CloudNativePG, Cluster.spec.instances,
not spec.replicas), and which instance to add or remove — and in what order
— is the engine operator's decision (CNPG removes the highest-ordinal
standby, never the primary).
Restate the real reason a stock HPA does not fit: it is topology-blind —
nothing encodes the synchronous-commit quorum floor (maxSyncReplicas + 1),
so it can drive the count below it, and there is no replication-lag gate.
The Flux argument is dropped from the HPA point; the accurate "a direct
patch to the operator CR is reverted by Flux" note under the direct-patch
alternative already covers that ground.
Signed-off-by: Alexey Artamonov <[email protected]>
Copy file name to clipboardExpand all lines: design-proposals/database-horizontal-autoscaling/README.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -32,7 +32,7 @@ A managed database in Cozystack is an `Application` in the aggregated `apps.cozy
32
32
33
33
> "My database is saturated with read traffic during business hours and idle at night, but I have to notice it, hand-edit `replicas`, and hope I picked the right number — and undo it later."
34
34
35
-
There is no automated way to add or remove read replicas under load. Manually reusing a stock `HorizontalPodAutoscaler` does not work here: HPA scales via the scale subresource on the operator CR, which **fights Flux** (reconciliation restores `replicas` from the application values), and HPA has no notion of database topology — it would happily remove a synchronous standby or scale while replication lag is unbounded.
35
+
There is no automated way to add or remove read replicas under load. A stock `HorizontalPodAutoscaler` does not fit: it only writes a replica count (for CloudNativePG, `Cluster.spec.instances`) and is blind to database topology. It has nothing to encode the synchronous-commit quorum floor, so it can drive the count below `maxSyncReplicas + 1` — where the operator either rejects the change or loses its write quorum — and it has no gate on replication lag, scaling on the load metric alone while standbys are arbitrarily behind. Which instance to add or remove, and in what order, is the engine operator's decision (CloudNativePG removes the highest-ordinal standby and never the primary); an autoscaler for stateful databases must own the count and the safety guardrails while leaving instance lifecycle to the operator.
36
36
37
37
## Goals
38
38
@@ -200,7 +200,7 @@ When several `metrics` are set, the desired count is the **maximum** of the per-
200
200
201
201
- **A controller inside `cozystack-controller`** instead of a standalone operator. It would reuse the existing binary, RBAC, and VictoriaMetrics helper, at the cost of coupling the autoscaler's lifecycle to the platform controller. Rejected in favor of a standalone operator for isolation and an independent release cadence; the logic can be moved later if desired.
202
202
- **Patching the engine CR / `HelmRelease` directly.** A direct patch to the operator CR is reverted by Flux. Patching `HelmRelease` values directly is possible but bypasses the supported surface. Note that the `Application` is a pure projection of the `HelmRelease` (rest.go converts both ways, `Values: app.Spec`), so patching the `Application`'s `replicas` **is** writing the same `HelmRelease` values — there is no background regeneration that would clobber it. The reasons to prefer the apps API are validation, label/lineage management, and it being the supported tenant surface, not clobber-avoidance.
203
-
- **Stock HPA + KEDA.** Rejected as the primary mechanism: it fights Flux, is topology-unaware, and offers no safe scale-down (quorum, lag, graceful removal). A KEDA/PromQL-style trigger could be reused *as a metric source inside* the operator only if the operator injects a mandatory tenant/namespace label matcher into every query and rejects any query it cannot constrain — never as raw tenant-supplied PromQL against shared vmselect.
203
+
- **Stock HPA + KEDA.** Rejected as the primary mechanism: it only writes a replica count and is topology-unaware — no synchronous-commit quorum floor, no replication-lag gate, and it cannot express which instance to remove (that is the engine operator's job). A KEDA/PromQL-style trigger could be reused *as a metric source inside* the operator only if the operator injects a mandatory tenant/namespace label matcher into every query and rejects any query it cannot constrain — never as raw tenant-supplied PromQL against shared vmselect.
204
204
- **Scaling the write path via sharding.** Out of scope: it requires data rebalancing (Cruise Control for Kafka, resharding for ClickHouse/MongoDB), which is an orchestrated procedure rather than a replica-count change.
0 commit comments