Skip to content

Commit 4db5652

Browse files
scooby87scooby87
authored
fix(postgres): garbage-collect init-job after completion (#3055) (#3069)
## Problem After deleting a `Postgres` app CR, the chart-managed init Job `<release>-init-job` and its completed Pod remain in the tenant namespace indefinitely (reported in #3055). The init Job is a Helm hook (`post-install,post-upgrade`) whose only delete policy is `before-hook-creation`. That policy deletes the previous hook only right before a new one is created (next upgrade) — it never fires on `helm uninstall`. With no `hook-succeeded` and no TTL, the completed Job and Pod leak. ## Root cause `packages/apps/postgres/templates/init-job.yaml` — the Job has no garbage-collection mechanism for its completed state. ## Fix Add `ttlSecondsAfterFinished: 120` to the Job spec. The native Kubernetes TTL controller then GCs the Job and its Pod ~2 minutes after completion, regardless of the Helm hook lifecycle. The init Job is a one-shot, idempotent reconciler of users/databases/roles (`init-script.yaml`) — once it has run it is not needed until the next upgrade (when Helm recreates it), so GC-ing the finished Job is safe. This approach deliberately **does not** touch the hook-delete-policy / hook-timeout semantics, which the `bootstrap.recovery` path depends on (see the header comment in `init-job.yaml`). Under `bootstrap.enabled` the Job is skipped entirely, so the change is inert there. ## Tests Added a `helm-unittest` suite `packages/apps/postgres/tests/init_job_cleanup_test.yaml`: - asserts the init Job renders `spec.ttlSecondsAfterFinished: 120` - asserts the init Job is skipped under `bootstrap.enabled: true` Verified locally with `helm template` (TTL present on default render; Job absent under bootstrap recovery). ## Test plan - [x] `helm template` renders `ttlSecondsAfterFinished: 120` on the init Job - [ ] `helm unittest packages/apps/postgres` green in CI - [ ] e2e `test-apps-postgres` green Fixes #3055 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Postgres bootstrap init Jobs now automatically clean up after completion using a post-finish TTL. * **Tests** * Added a new integration test suite to verify the init Job renders with the expected cleanup TTL, and that the init Job is skipped when bootstrap is enabled. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents 49555e5 + af1cc79 commit 4db5652

2 files changed

Lines changed: 53 additions & 0 deletions

File tree

‎packages/apps/postgres/templates/init-job.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ metadata:
2020
"helm.sh/hook-weight": "-5"
2121
"helm.sh/hook-delete-policy": before-hook-creation
2222
spec:
23+
ttlSecondsAfterFinished: 3600
2324
template:
2425
metadata:
2526
name: {{ .Release.Name }}-init-job

‎packages/apps/postgres/tests/init_job_cleanup_test.yaml‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
suite: init-job is garbage-collected after completion
2+
3+
# Locks in cozystack/cozystack#3055: the post-install/post-upgrade init Job
4+
# is a one-shot reconciler of users/databases/roles. Without a TTL it lingers
5+
# in the namespace forever after the Postgres CR is deleted (its hook-delete-policy
6+
# is 'before-hook-creation' only, which never fires on uninstall). A native
7+
# ttlSecondsAfterFinished lets the Kubernetes TTL controller GC the Job and its
8+
# Pod ~1 hour after it completes, regardless of Helm hook lifecycle — without
9+
# touching the hook-timeout semantics the recovery flow depends on. 1h leaves a
10+
# debugging window for a failed init Job before it is collected.
11+
12+
# Both templates are loaded so the Job's checksum/config annotation can resolve
13+
# its `include ".../init-script.yaml"`. Each test then pins assertions to a
14+
# single template via `template:` so the per-template document set is exact.
15+
templates:
16+
- templates/init-job.yaml
17+
- templates/init-script.yaml
18+
19+
release:
20+
name: pg-test
21+
namespace: tenant-test
22+
23+
set:
24+
_cluster:
25+
scheduling: {}
26+
users:
27+
app:
28+
password: secret
29+
30+
tests:
31+
- it: "init-job sets ttlSecondsAfterFinished so Kubernetes GCs it after the run"
32+
template: templates/init-job.yaml
33+
asserts:
34+
- hasDocuments:
35+
count: 1
36+
- isKind:
37+
of: Job
38+
- equal:
39+
path: metadata.name
40+
value: pg-test-init-job
41+
- equal:
42+
path: spec.ttlSecondsAfterFinished
43+
value: 3600
44+
45+
- it: "init-job is skipped entirely under bootstrap recovery"
46+
template: templates/init-job.yaml
47+
set:
48+
bootstrap:
49+
enabled: true
50+
asserts:
51+
- hasDocuments:
52+
count: 0

0 commit comments

Comments
 (0)