e2e: kafka teardown overruns the 5m delete budget by seconds #4260

Describe the bug

verify-pvc-reclaim-on-delete deletes the Kafka CR and waits for the deletion to finish. The wait hits the 5m default and fails the suite. Thirteen seconds later the object is gone, so the budget and the teardown sit close enough that a run can land on either side.

Environment

• Cozystack: main
• In-tree e2e lane, run 34891318171

To Reproduce

Run the full chainsaw suite. The kafka suite fails at verify-pvc-reclaim-on-delete after about 455s.

Expected behaviour

The teardown finishes inside the budget, or the budget reflects what a Kafka teardown costs.

Actual behaviour

The delete was accepted at 22:08:08 and the wait gave up exactly five minutes later:

22:08:08 | kafka | verify-pvc-reclaim-on-delete | DELETE | OK    | apps.cozystack.io/v1alpha1/Kafka @ tenant-test/test
22:13:08 | kafka | verify-pvc-reclaim-on-delete | DELETE | ERROR | apps.cozystack.io/v1alpha1/Kafka @ tenant-test/test
         === ERROR
         context deadline exceeded

By the time the catch block ran at 22:13:21, both the CR and its HelmRelease were gone:

Error from server (NotFound): helmreleases.helm.toolkit.fluxcd.io "kafka-test" not found
kafkas.apps.cozystack.io "test" not found

Logs

The step declares no timeout of its own, so it takes the suite default:

$ sed -n '95,103p' hack/e2e-chainsaw/kafka/chainsaw-test.yaml
  - name: verify-pvc-reclaim-on-delete
    try:
    - delete:
        ref:
          apiVersion: apps.cozystack.io/v1alpha1
          kind: Kafka
          name: test
    - error:
        timeout: 2m

The default is five minutes, picked with finalizer-draining CRs in mind:

$ sed -n '21,28p' hack/e2e-chainsaw/.chainsaw.yaml
  timeouts:
    apply: 1m
    assert: 5m
    # delete/cleanup are generous: finalizer-draining CRs (VMDisk, FoundationDB,
    # Tenant child namespaces) can take minutes to fully reclaim their backing
    # resources, and Chainsaw waits for the delete to complete during cleanup.
    delete: 5m
    cleanup: 5m

Additional context

One observation, one run, no rate claimed. The log does not say whether the teardown finished just after the deadline or finished earlier and the wait missed it. Thirteen seconds fits both, and the catch block's own kubectl calls are the only evidence the object was gone.

Worth measuring before the number moves. Kafka drags Zookeeper PVCs and a HelmRelease behind it, so if a teardown near five minutes is normal here then the budget is wrong rather than the run unlucky. A few runs with the delete instrumented would say which, and that is cheaper than moving a timeout somebody set deliberately.

Checklist

• I have checked the documentation
• I have searched for similar issues
• I have included all required information
• I have provided clear steps to reproduce
• I have included relevant logs