Tenant quota: admission and the tenantquota controller disagree on a child whose namespace was never rendered #5314

Describe the bug

Hierarchical tenant quotas are counted in two places. The two places disagree about a child tenant whose HelmRelease never rendered its namespace.

• Admission (siblingDeclaredQuotas, pkg/registry/apps/application/quota.go#L136) sums the resourceQuotas from the values of every sibling tenant HelmRelease, whether it is Ready or not.
• The tenantquota controller (snapshot, internal/controller/tenantquota/reconciler.go#L170) takes a child's budget from the tenant-quota ResourceQuota rendered in the child's namespace. A child whose namespace was never created therefore has an empty Declared. ComputePools then treats it as an unbounded member of the parent pool instead of a carve-out.

For such a child, admission subtracts the declared quota from the parent's remaining budget, while the controller subtracts nothing.

Environment

• Cozystack v1.6.4 (observed). The code paths above are unchanged in v1.7.0 and on main (56224a1). Since v1.7.0, quota.go only changed the usage key mapping (54695fc), which does not touch sibling counting.
• Provider: on-prem

To Reproduce

1. Give a parent tenant a bounded quota, e.g. resourceQuotas.storage: 6Ti.
2. Create child tenants under it with declared quotas. Let some of them fail before their namespace exists: the HelmRelease is not Ready and tenant-<parent>-<child> was never created. On the install below this happened to 9 children, for reasons unrelated to quotas.
3. Compare the admission error for a new child with the controller-owned tenant-quota-allocated in the parent namespace.

Expected behaviour

Admission and the controller agree on how much of the parent budget is carved out by children.

Actual behaviour

On one install, a parent with storage: 6Ti has 27 child tenants. 9 of them never rendered a namespace: 6 × 2Gi + 3 × 200Gi = 612Gi declared.

• Admission rejects a new 200Gi child:

  spec.resourceQuotas[storage]: Forbidden: requested 200Gi exceeds the parent tenant's remaining "storage" quota of 122Gi (parent allows 6Ti, 5862Gi already allocated to sibling tenants, 160Gi currently in use by the parent pool)
  

  5862Gi = all 27 children.
• The controller sets tenant-quota-allocated in the parent namespace to requests.storage: 894Gi, i.e. 6Ti − 5250Gi. 5250Gi = only the 18 children that have a rendered tenant-quota.

The 612Gi gap is exactly the declared quota of the children without a namespace.

Consequences:

1. Phantom reservation at admission. A child that runs nothing and holds no resources still blocks new siblings. Operators see "quota exhausted" while usage is low and the controller's view has room.
2. The parent pool can be overcommitted. The controller lets the parent's own workloads grow to 894Gi, while admission assumes 6Ti − 5862Gi = 282Gi is left for the parent pool and new children. If the failed children later render their namespaces, the carve-outs plus the parent's usage can exceed the parent budget. recordOvercommit would report it, but nothing prevents it.

Logs

See the admission message above. The controller side:

$ kubectl -n tenant-<parent> get resourcequota tenant-quota tenant-quota-allocated \
    -o custom-columns=NAME:.metadata.name,HARD:.spec.hard.requests\\.storage,USED:.status.used.requests\\.storage
NAME                     HARD   USED
tenant-quota             6Ti    160Gi
tenant-quota-allocated   894Gi  160Gi

Additional context

Either side could move. Two options, for whoever owns this:

• the controller falls back to the child HelmRelease's declared values when the rendered tenant-quota is absent, so a declared-but-unrendered child stays a carve-out (this matches admission and errs on the safe side);
• or admission skips siblings whose namespace does not exist, matching the controller (this frees the phantom reservation but keeps the overcommit window open).

Checklist

• I have checked the documentation
• I have searched for similar issues
• I have included all required information
• I have provided clear steps to reproduce
• I have included relevant logs