Skip to content

Clearing using ExternalTaskMarker will not activate external DagRuns #14260

@eejbyfeldt

Description

@eejbyfeldt

Apache Airflow version:

2.0.1

What happened:

When clearing task across dags using ExternalTaskMarker the dag state of the external DagRun is not set to active. So cleared tasks in the external dag will not automatically start if the DagRun is a Failed or Succeeded state.

What you expected to happen:

The external DagRun run should also be set to Running state.

How to reproduce it:

Clear tasks in an external dag using an ExternalTaskMarker.

Anything else we need to know:

Looking at the code is has:

airflow/airflow/models/dag.py

Lines 1323 to 1335 in b23fc13

if do_it:
clear_task_instances(
tis,
session,
dag=self,
activate_dag_runs=False, # We will set DagRun state later.
)
self.set_dag_runs_state(
session=session,
start_date=start_date,
end_date=end_date,
state=dag_run_state,
)

It seems like it intentionally calls the dag member method set_dag_run_state instead of letting the helper function clear_task_insntances set the DagRun state. But the member method will only change the state of DagRuns of dag where the original task is, while I believe clear_task_instances would correctly change the state of all involved DagRuns.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions