Repository navigation
App chart Jobs without resources inherit the tenant LimitRange default #4868
Description
Some Jobs in the app charts run without resources of their own. In a tenant with resourceQuotas they get the LimitRange default from packages/apps/tenant/templates/quota.yaml, which is 128Mi of memory and 250m of CPU. Without quotas they run BestEffort.
This matters for kubectl apply on an object that already exists. To build the patch, kubectl downloads the cluster's whole OpenAPI document. I measured clastix/kubectl:v1.32 in a kind cluster loaded with the 374 CRDs of an e2e cluster:
| Command | Peak memory |
|---|---|
kubectl apply of a KafkaNodePool that already exists |
about 300Mi |
kubectl apply that creates it |
about 30Mi |
kubectl apply --server-side of the same object |
33Mi |
kubectl apply --openapi-patch=false of the same object |
about 35Mi |
kubectl patch, kubectl get |
25Mi |
The Kafka migration hook hit this on upgrade: it was OOM-killed at 128Mi and the upgrade failed with "pre-upgrade hooks failed". #4865 fixes that one. These Jobs in other charts have no resources:
| Chart | Template | Kind | What it runs | Risk |
|---|---|---|---|---|
| vm-disk | templates/pvc-resize-hook.yaml |
pre-install, pre-upgrade hook | kubectl patch on the PVC |
low, a patch stays near 25Mi |
| postgres | templates/init-job.yaml |
post-install, post-upgrade hook | psql scripts |
unmeasured |
| kubernetes | templates/talos/bootstrap-token-tenant-job.yaml |
Job, not a hook | client-side kubectl apply against the tenant cluster |
low today: the tenant cluster has few CRDs, but the same pattern grows with them |
None of them is failing now as far as I know. I'd still give each one explicit requests and limits, and pass --openapi-patch=false wherever a Job client-side applies objects that may already exist. For custom resources it builds the same merge patch without the download. --server-side is as small, but it changes which fields a later apply removes. By the numbers above, the hooks that already set resources fit their limits: against the management cluster they only delete, get or patch, and the one client-side apply (kubernetes/templates/oidc-rbac-job.yaml) targets the tenant cluster with a 256Mi limit.
Activity
fix(kafka): pass the container limits to the Kafka JVMs (#4865)