Skip to content

Commit 324815a

Browse files
committed
docs(dha): correct the stock-HPA motivation (topology, not victim selection)
The motivation and the "Stock HPA + KEDA" alternative claimed HPA "would happily remove a synchronous standby." That misattributes the mechanism: HPA only writes a replica count (for CloudNativePG, Cluster.spec.instances, not spec.replicas), and which instance to add or remove — and in what order — is the engine operator's decision (CNPG removes the highest-ordinal standby, never the primary). Restate the real reason a stock HPA does not fit: it is topology-blind — nothing encodes the synchronous-commit quorum floor (maxSyncReplicas + 1), so it can drive the count below it, and there is no replication-lag gate. The Flux argument is dropped from the HPA point; the accurate "a direct patch to the operator CR is reverted by Flux" note under the direct-patch alternative already covers that ground. Signed-off-by: Alexey Artamonov <[email protected]>
1 parent a3749c0 commit 324815a

1 file changed

Lines changed: 2 additions & 2 deletions

File tree

  • design-proposals/database-horizontal-autoscaling

‎design-proposals/database-horizontal-autoscaling/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ A managed database in Cozystack is an `Application` in the aggregated `apps.cozy
3232

3333
> "My database is saturated with read traffic during business hours and idle at night, but I have to notice it, hand-edit `replicas`, and hope I picked the right number — and undo it later."
3434
35-
There is no automated way to add or remove read replicas under load. Manually reusing a stock `HorizontalPodAutoscaler` does not work here: HPA scales via the scale subresource on the operator CR, which **fights Flux** (reconciliation restores `replicas` from the application values), and HPA has no notion of database topology — it would happily remove a synchronous standby or scale while replication lag is unbounded.
35+
There is no automated way to add or remove read replicas under load. A stock `HorizontalPodAutoscaler` does not fit: it only writes a replica count (for CloudNativePG, `Cluster.spec.instances`) and is blind to database topology. It has nothing to encode the synchronous-commit quorum floor, so it can drive the count below `maxSyncReplicas + 1` — where the operator either rejects the change or loses its write quorum — and it has no gate on replication lag, scaling on the load metric alone while standbys are arbitrarily behind. Which instance to add or remove, and in what order, is the engine operator's decision (CloudNativePG removes the highest-ordinal standby and never the primary); an autoscaler for stateful databases must own the count and the safety guardrails while leaving instance lifecycle to the operator.
3636

3737
## Goals
3838

@@ -200,7 +200,7 @@ When several `metrics` are set, the desired count is the **maximum** of the per-
200200

201201
- **A controller inside `cozystack-controller`** instead of a standalone operator. It would reuse the existing binary, RBAC, and VictoriaMetrics helper, at the cost of coupling the autoscaler's lifecycle to the platform controller. Rejected in favor of a standalone operator for isolation and an independent release cadence; the logic can be moved later if desired.
202202
- **Patching the engine CR / `HelmRelease` directly.** A direct patch to the operator CR is reverted by Flux. Patching `HelmRelease` values directly is possible but bypasses the supported surface. Note that the `Application` is a pure projection of the `HelmRelease` (rest.go converts both ways, `Values: app.Spec`), so patching the `Application`'s `replicas` **is** writing the same `HelmRelease` values — there is no background regeneration that would clobber it. The reasons to prefer the apps API are validation, label/lineage management, and it being the supported tenant surface, not clobber-avoidance.
203-
- **Stock HPA + KEDA.** Rejected as the primary mechanism: it fights Flux, is topology-unaware, and offers no safe scale-down (quorum, lag, graceful removal). A KEDA/PromQL-style trigger could be reused *as a metric source inside* the operator only if the operator injects a mandatory tenant/namespace label matcher into every query and rejects any query it cannot constrain — never as raw tenant-supplied PromQL against shared vmselect.
203+
- **Stock HPA + KEDA.** Rejected as the primary mechanism: it only writes a replica count and is topology-unaware — no synchronous-commit quorum floor, no replication-lag gate, and it cannot express which instance to remove (that is the engine operator's job). A KEDA/PromQL-style trigger could be reused *as a metric source inside* the operator only if the operator injects a mandatory tenant/namespace label matcher into every query and rejects any query it cannot constrain — never as raw tenant-supplied PromQL against shared vmselect.
204204
- **Scaling the write path via sharding.** Out of scope: it requires data rebalancing (Cruise Control for Kafka, resharding for ClickHouse/MongoDB), which is an orchestrated procedure rather than a replica-count change.
205205

206206
---

0 commit comments

Comments
 (0)