Skip to content

When triggering a paused DAG, ensure the manually triggered run starts before a scheduled run #55675

Description

@brki

Description

For the description below, assume that the DAG:

  • is scheduled to run once a day
  • has been paused more than one day
  • has max_active_runs=1
  • was not running when it was paused (e.g. all tasks are success or failed)

When triggering a paused DAG, it would be nice if the manually triggered run starts before any scheduled run.

If the DAG has parameters, there is a "Unpause DAG when triggered" toggle, but this doesn't help.

This is the behavior currently seen, running with the official Helm chart, using the Celery scheduler:

  • With "Unpause DAG when triggered" = true: regularly scheduled run starts running, the manually triggered run can run only after the other one completes (naturally, or because marked success / failure)
  • With "Unpause DAG when triggered" = false: only the manually triggered run is scheduled - until the DAG is unpaused, at which time the regularly scheduled run starts running, and the manually triggered run can run only after the other one completes (naturally, or because marked success / failure)

Use case/motivation

We have an Integration environment, but most of the DAGs are usually paused on Integration. Some of the DAGs process a lot of data, and take a long time to run. These DAGs are run on Integration only occasionally, for example when testing changes. These DAGs often have parameters to reduce the amount of data processed (e.g. only process the list of ids (1,2,3), or only process the source data within a given date/time range).

It's an annoyance for us, and counterintuitive, that when we trigger a paused DAG, the manually triggered run, with the parameters we have specified, is not the one that starts running - it's the regularly scheduled DAG run that goes first (obviously, without the parameters that have been specified). We have to then mark the regularly scheduled run as fail / success, before our parameterized manual run starts.

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions