Repository navigation
Commit b323d5d
authored
fix(kafka): keep the pre-delete hook's credentials as long as its Job (cozystack#4280)
## What this PR does
Fixes cozystack#4276. Since cozystack#3938 landed, no Kafka release finishes deleting: the
pre-delete hook cannot reach the apiserver, exits non-zero, and a
non-zero pre-delete hook returns from `Uninstall.Run` before a single
resource is removed. It is the inverse of what the hook was written for.
cozystack#3793 was one release that would not delete; this made every release
refuse to.
The cause is `--request-timeout=20s` on the delete call. Any non-zero
value for that flag makes kubectl skip its in-cluster credentials
altogether: client-go consults them only while the assembled config
still equals the default, and the flag lands in that config, so the
branch that would load the ServiceAccount token never runs. kubectl
talks to its built-in default cluster, `localhost:8080`, is refused, and
the script reads the refusal as a failed delete.
Measured on the pinned image rather than argued from documentation. With
the token and `ca.crt` mounted and `KUBERNETES_SERVICE_HOST` set, the
same delete call reaches the cluster with no flag, goes to
`localhost:8080` with `--request-timeout=20s`, goes to `localhost:8080`
with `1m`, and reaches the cluster again with `0`.
The tree agrees. None of the three hooks that work pass it:
`apps/kubernetes` names it in a comment only, `apps/harbor` and
`apps/mariadb` never mention it, and
`apps/tenant/templates/cleanup-job.yaml` records that its DELETE calls
"carry kubectl's default `--request-timeout` of 0". The watch is still
bounded by `--timeout`, and the Job by `activeDeadlineSeconds`, which is
that same shape.
The second commit was a separate change found on the way, and review
turned it around. It matched the credentials to the Job on
`before-hook-creation` alone, and a ServiceAccount, Role or RoleBinding
has nothing like `ttlSecondsAfterFinished` to fall back on: helm removes
a hook resource only when a policy says so, and `before-hook-creation`
fires on a next activation that after the last uninstall never comes.
All three would have sat in the tenant namespace for the life of the
cluster, still granting delete over every KafkaTopic in it. They now
carry `before-hook-creation,hook-succeeded`, the shape `apps/clickhouse`
and `apps/kubernetes-nodes` already use, while the Job keeps
`before-hook-creation` alone because the script exits 0 on the degraded
path too and the Job's log is the only record that a finalizer outlived
the wait. Reaping the credentials cannot strand a running pod: helm
applies `hook-succeeded` only once the hook is terminal, and
`activeDeadlineSeconds: 270` ends the Job inside the 5m uninstall hook
timeout. Neither of these caused cozystack#4276, and a cluster run confirmed as
much.
Both are pinned. Restoring the flag turns the suite red, with `0` as
well as with `20s`; the assertion matches `--request-timeout=` rather
than the bare name, because the script's own comment names the flag.
Dropping the reaping policy from the ServiceAccount alone, or from the
Role alone, turns it red too, so the three are three assertions rather
than one.
Worth stating plainly, because it is the reason this reached `main`:
none of this is visible in a rendered manifest. The asymmetry and the
flag both rendered cleanly, passed 37 helm-unittest cases and 5 bats
cases through two rounds of review and per-branch mutation testing, and
only a live uninstall showed either. The e2e lane never ran on cozystack#3938
while it was open, because its workflows sat at `action_required` on all
five heads a fork PR had.
### Screenshots
Not a UI change.
### Downstream repositories
Walked the trigger map against the diff. This changes hook annotations
and test assertions inside `packages/apps/kafka`; no values key, schema,
API, default or naming contract moves, and the app itself is neither
added nor renamed.
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
### Release note
```release-note
fix(kafka): deleting a Kafka release no longer wedges on a pre-delete hook that cannot reach the apiserver
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved Kafka app uninstall cleanup by removing an unnecessary
request-timeout constraint while retaining existing overall timeout
safeguards.
- Updated cleanup resource policies to remove failed-hook retention and
preserve resources through successful hook completion, supporting
reliable repeated uninstall attempts.
- **Tests**
- Added coverage for cleanup command timeout behavior and resource
deletion policies.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->2 files changed
Lines changed: 59 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
65 | 65 | | |
66 | 66 | | |
67 | 67 | | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
68 | 81 | | |
69 | 82 | | |
70 | 83 | | |
71 | 84 | | |
72 | | - | |
73 | | - | |
| 85 | + | |
74 | 86 | | |
75 | 87 | | |
76 | 88 | | |
| |||
100 | 112 | | |
101 | 113 | | |
102 | 114 | | |
103 | | - | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
104 | 120 | | |
105 | 121 | | |
106 | 122 | | |
| |||
111 | 127 | | |
112 | 128 | | |
113 | 129 | | |
114 | | - | |
| 130 | + | |
| 131 | + | |
115 | 132 | | |
116 | 133 | | |
117 | 134 | | |
| |||
126 | 143 | | |
127 | 144 | | |
128 | 145 | | |
129 | | - | |
| 146 | + | |
| 147 | + | |
130 | 148 | | |
131 | 149 | | |
132 | 150 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
54 | 54 | | |
55 | 55 | | |
56 | 56 | | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
57 | 70 | | |
58 | 71 | | |
59 | 72 | | |
| |||
69 | 82 | | |
70 | 83 | | |
71 | 84 | | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
72 | 100 | | |
73 | 101 | | |
74 | 102 | | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
75 | 107 | | |
76 | 108 | | |
77 | 109 | | |
| |||
82 | 114 | | |
83 | 115 | | |
84 | 116 | | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
85 | 121 | | |
86 | 122 | | |
87 | 123 | | |
| |||
0 commit comments