Repository navigation
Conversation
693f3f6 to
edb252d
Compare
|
did you read this -> #71384 (comment) ? |
|
Thanks for the review. Yes, I had read that comment, but I had not compared #71384's diff closely enough. Looking at it now, this PR makes essentially the same core behavior change. Sorry about that. This is a pause-semantics decision, not just an implementation detail. Should we intentionally change the existing pause behavior, or keep pause as-is and make "stop starting new Dag runs while existing runs drain" a separate feature? Maybe we can let the maintainers decide which direction they prefer here. |
potiuk
added
the
closed because of open PR limit
|
Hello @fat-catTW - thank you for your contributions to Apache Airflow! The Airflow community has introduced a limit of 5 open pull requests at a time for contributors without write access to the repository. You currently have 17 open pull requests, so - as a one-time step of introducing the limit - we closed the ones where maintainers have not engaged yet:
These pull requests stay open because maintainers are already engaged in them - they count towards your limit:
This is not a judgement of you or of your changes. We never told contributors before that opening many pull requests at once was a problem, so there is nothing to feel bad about - and nothing is lost: your branches, commits and the review history stay where they are. What we ask you to do is to make your first prioritization decision: choose which of the pull requests above matter most to you, and reopen them (up to 5 open at a time, including the ones still open) with the "Reopen pull request" button or While your pull requests are waiting for review, the most valuable thing you can do is help in other ways - reviewing other contributors' pull requests, helping with issues, and taking part in the discussions on the devlist and Slack. Why we introduced the limit, what it means for you and how to reopen or restore a pull request is explained in https://github.com/apache/airflow/blob/main/contributing-docs/32_open_pull_request_limit.rst. Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting |
Why
Pausing a Dag currently blocks more than new Dag run creation. If a Dag is paused while a Dag run is already
running, the scheduler stops examining that run and stops queueing its remaining scheduled task instances.That leaves the run silently stuck in
running, with remaining tasks staying inscheduledand never reaching the executor. Unpausing fixes the stuck run, but also allows new runs to start, which is not always what the operator intended.relates: #71381
Solution
Keep pause behavior for new work, but let already-running Dag runs continue.
Queued Dag runs still require an unpaused Dag before they can move to
running. The scheduler now only removes the pause gate from paths that operate on Dag runs already inrunningstate:This preserves pause as "do not start new runs" while allowing in-flight runs to finish.
Testing
uv run ruff check --fix airflow-core/src/airflow/models/dagrun.py airflow-core/src/airflow/jobs/scheduler_job_runner.py airflow-core/tests/unit/models/test_dagrun.py airflow-core/tests/unit/jobs/test_scheduler_job.pyuv run --project airflow-core pytest airflow-core/tests/unit/models/test_dagrun.py::TestDagRun::test_next_dagruns_to_examine_paused_dag airflow-core/tests/unit/jobs/test_scheduler_job.py::TestSchedulerJob::test_execute_task_instances_for_paused_running_dagrun -qgit diff --checkWas generative AI tooling used to co-author this PR?
[X]Yes (please specify the tool below)Generated-by: [Codex] following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.