Skip to content

Fix Celery worker liveness probe hostname lookup#67471

Merged
jscheffl merged 3 commits into
apache:mainfrom
bmanan7:mb/fix-celery-liveness-hostname-upstream
May 26, 2026
Merged

Fix Celery worker liveness probe hostname lookup#67471
jscheffl merged 3 commits into
apache:mainfrom
bmanan7:mb/fix-celery-liveness-hostname-upstream

Conversation

@bmanan7

@bmanan7 bmanan7 commented May 25, 2026

Copy link
Copy Markdown

Description

The default Celery worker liveness probe currently builds the Celery node name with the OS-level hostname executable:

celery inspect ping -d celery@$(hostname)

Some minimal container images do not include that executable, which makes the probe fail before it can check whether the Celery worker is alive.

This keeps the same behavior of targeting the worker container's hostname, but reads it with Python's standard socket.gethostname() instead:

celery inspect ping -d celery@$(python -c 'import socket; print(socket.gethostname())')

The probe already depends on Python to run Celery, so this avoids adding another OS binary requirement. Kubernetes also documents that the container hostname is available through the gethostname function call: https://kubernetes.io/docs/concepts/containers/container-environment/

Testing

prek run --files chart/templates/workers/worker-deployment.yaml chart/tests/helm_tests/airflow_core/test_worker.py
Passed
env PYTHONPATH=chart/tests /private/tmp/airflow-chart-test-venv/bin/python -m pytest chart/tests/helm_tests/airflow_core/test_worker.py
377 passed in 168.12s

Was generative AI tooling used to co-author this PR?
  • Yes - OpenAI assistant

Generated-by: OpenAI assistant following https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions

@boring-cyborg boring-cyborg Bot added the area:helm-chart Airflow Helm Chart label May 25, 2026
@boring-cyborg

boring-cyborg Bot commented May 25, 2026

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: [email protected]
    Slack: https://s.apache.org/airflow-slack

@bmanan7 bmanan7 marked this pull request as ready for review May 25, 2026 13:32
Comment thread chart/newsfragments/67471.bugfix.rst Outdated
Comment thread chart/tests/helm_tests/airflow_core/test_worker.py Outdated
@jscheffl jscheffl added the backport-to-chart/v1-2x-test Automatic backport to chart 1.2x maintenance branch label May 26, 2026
@jscheffl jscheffl requested a review from henry3260 May 26, 2026 08:52

@henry3260 henry3260 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@jscheffl jscheffl merged commit eacbd69 into apache:main May 26, 2026
102 checks passed
@boring-cyborg

boring-cyborg Bot commented May 26, 2026

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: chart/v1-2x-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
chart/v1-2x-test PR Link

jscheffl pushed a commit that referenced this pull request May 26, 2026
…67471) (#67558)

* Fix Celery worker liveness probe hostname lookup

* Add chart newsfragment for Celery liveness probe fix

* Address review feedback for Celery liveness probe fix
(cherry picked from commit eacbd69)

Co-authored-by: Manan Bhatt <[email protected]>
@jscheffl jscheffl added this to the Airflow Helm Chart 1.22.0 milestone May 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:helm-chart Airflow Helm Chart backport-to-chart/v1-2x-test Automatic backport to chart 1.2x maintenance branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants