Context

Previously, the worker log groomer sidecar only rendered when workers.persistence.enabled was true (i.e., StatefulSet mode). This prevented using the log groomer on Deployments with emptyDir volumes for logs.

This change removes the persistence requirement from the log groomer condition. Now the log groomer renders whenever logGroomerSidecar.enabled is true, regardless of the persistence setting.

This is useful for deployments that:

• Use emptyDir for local logs with remote logging (S3, GCS, etc.)
• Want to clean up local log files to prevent ephemeral storage exhaustion
• Prefer Deployments over StatefulSets for workers

Users who don't want the log groomer can disable it by setting workers.logGroomerSidecar.enabled: false.

Backward Compatibility

This change may add a new sidecar container to existing worker Deployments. Specifically:

Impact: Users with persistence.enabled: false who left logGroomerSidecar.enabled or the
default (true) will now get the log groomer sidecar. This will cause worker pods to be recreated on
upgrade.

Mitigation: Users who don't want the log groomer can explicitly set
workers.logGroomerSidecar.enabled: false.

Rationale:

• This is much cleaner solution (without introducing a new argument, etc.).
• The log groomer being silently disabled when persistence is off could be considered a
  bug—users who enable the log groomer expect it to run. The sidecar is lightweight and helps prevent
  ephemeral storage exhaustion.

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.