Skip to content

Pausing a DAG stalls already-running DagRuns instead of only blocking new ones #71381

Open

Description

@juteasl

Description

When a DAG is paused while it has an active (running) DagRun — e.g. one started via manual trigger before the pause, or a manual trigger on an already-paused DAG that got an initial burst of tasks dispatched — the scheduler stops scheduling further task instances for that run. The run doesn't fail and logs no error; it simply stalls indefinitely in running state with remaining tasks stuck in scheduled, queued_when=None, never sent to the executor.

This happens because is_paused is checked as a single gate that governs both "should new DagRuns be created" and "should ready task instances in existing DagRuns be scheduled." The two are conflated.

Use case/motivation

Pause is a reasonable way to say "don't start new runs of this DAG." It's not intuitive that it also freezes runs already in flight — especially since:

  • No error, warning, or DAG/task state reflects why the run stopped progressing.
  • A manually triggered run against a paused DAG appears to start successfully (initial tasks execute), masking the problem until the run has been silently stuck for several minutes.
  • The eventual fix (unpause) has a side effect the operator may not want: it also releases scheduling for new runs of that DAG, when the operator's only goal was to unstick the one in-flight run.

Suggested change

Preserve pre-run pause semantics: is_paused should gate DagRun creation, not task-instance scheduling for DagRuns already in running state. A DagRun that starts (however it started) while unpaused, or that gets tasks dispatched before a pause takes effect, should be allowed to run to completion. If the DAG is still paused when the run finishes, subsequent runs are blocked as expected — pause state before and after the run is preserved; only the in-flight run is exempted.

At minimum, surfacing this in the UI/logs (e.g., a task/run banner: "this DagRun's DAG is paused; scheduling is suspended") would prevent the silent-stall confusion, even if the gating behavior itself is kept as-is by design.

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Activity

  1. boring-cyborg commented on Aug 10, 2026

    @boring-cyborg

    Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions