Skip to content

Short-circuit in a mapped task group lets later all_done and none_failed tasks run #74175

Closed

Description

@luc-pimentel

Under which category would you file this issue?

Airflow Core

Apache Airflow version

main (as of 2026-10-03); also reproduced on 3.1.8

What happened and how to reproduce it?

Inside a mapped task group, a short-circuit task that returns False skips only the task right after it. Later tasks of the same map index still run if their trigger rule accepts a skipped upstream (all_done, none_failed), even with the default ignore_downstream_trigger_rules=True.

import pendulum

from airflow.sdk import dag, task, task_group


@dag(schedule=None, start_date=pendulum.datetime(2026, 1, 1, tz="UTC"), catchup=False)
def short_circuit_mapped_group():
    @task_group
    def group(value):
        @task.short_circuit  # ignore_downstream_trigger_rules=True (default)
        def gate(v):
            return v

        @task
        def a():
            pass

        @task(trigger_rule="all_done")  # "none_failed" behaves the same
        def b():
            pass

        gate(value) >> a() >> b()

    group.expand(value=[True, False])


short_circuit_mapped_group()

Steps: run airflow standalone (SQLite, LocalExecutor), trigger the Dag and check the task instances of map index 1. airflow dags test short_circuit_mapped_group gives the same states.

Map index 1 (gate returns False):

Build group.a[1] group.b[1], all_done group.b[1], none_failed
3.1.8 skipped success success
main skipped success success

Map index 0 runs everything, as expected. Controls on the same builds:

  • The same chain outside a mapped task group: a and b are skipped.
  • ignore_downstream_trigger_rules=False in the mapped group: b[1] runs, the same as with True.
  • b with the default all_success: b[1] is skipped, because the skip propagates through a.

What you think should happen instead?

group.b[1] should be skipped. The ShortCircuitOperator docs say that with ignore_downstream_trigger_rules=True, "all downstream tasks are skipped without considering the trigger_rule defined for tasks". That holds outside a mapped task group, but inside one the flag currently has no effect.

The XCom that gate writes for map index 1 lists both group.a and group.b. For a mapped task instance, SkipMixin._set_state_to_skipped in task-sdk/src/airflow/sdk/bases/skipmixin.py does not skip anything itself and leaves it to NotPreviouslySkippedDep, which only reads the decision of direct upstream tasks, so only group.a picks it up.

Operating System

macOS 26.2

Deployment

Virtualenv installation

Apache Airflow Provider(s)

standard

Versions of Apache Airflow Providers

apache-airflow-providers-standard 1.12.0 (with 3.1.8); main installed from source.

Official Helm Chart version

Not Applicable

Kubernetes Version

Not Applicable

Helm Chart configuration

Not Applicable

Docker Image customizations

Not Applicable

Anything else?

It happens every time. The main build I tested already includes #73959 and #73960, the two fixes from #55225; this case is not covered by either. On 3.2.0 to 3.3.2, nothing inside a mapped task group is skipped at all, not even group.a[1], which #73959 fixed on main. #73960 covers a different case: an unmapped short-circuit with an expanded mapped downstream task.


Drafted-by: Claude Code (Opus 5.5); reviewed by @luc-pimentel before posting

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Activity

  1. added
    kind:bugThis is a clearly a bug
    needs-triagelabel for new issues that we didn't triage yet
    on Oct 3, 2026
  2. akshat-lakhera commented on Oct 4, 2026

    @akshat-lakhera

    Hey @luc-pimentel, I would like to work on this can you assign this to me

  3. shahar1 commented on Oct 4, 2026

    @shahar1
    Contributor

    Hey @luc-pimentel, I would like to work on this can you assign this to me

    Currently we don't assign contributors to issues according to policy (with some exceptions).
    Feel free to create a PR without assignment.

  4. anzinmhd commented on Oct 4, 2026

    @anzinmhd
    Contributor

    I’ve reproduced the issue and identified the cause in NotPreviouslySkippedDep.

    The fix updates it to consider all upstream relatives so that skip decisions from a ShortCircuitOperator inside a mapped task group are correctly respected by transitive downstream tasks.

    I’ve also added a regression test and verified that the reported behavior is fixed.

    A PR is being opened with the fix: Fix mapped task group short circuit skipping.

    If this approach needs any changes or there’s a better way to address the issue, I’m happy to discuss and adjust the implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions