[airflow] Implement airflow-xcom-pull-in-template-string (AIR201) by Dev-iL · Pull Request #23583 · astral-sh/ruff

Dev-iL · 2026-02-26T14:27:51Z

Summary

Implements rule AIR004 (airflow-xcom-pull-in-template-string) that detects Airflow operator/sensor keyword arguments using a Jinja template string containing a single xcom_pull call (e.g., "{{ ti.xcom_pull(task_ids='some_task') }}") and suggests replacing it with the .output attribute on the task object (e.g., some_task.output).

Using .output instead of xcom_pull template strings:

Makes task dependencies explicit and visible to the DAG parser
Provides better IDE support (autocompletion, go-to-definition)
Is the recommended pattern for both traditional operators and TaskFlow API (@task-decorated functions)

What the rule flags

from airflow.operators.python import PythonOperator
from airflow.operators.bash import BashOperator

task_1 = PythonOperator(task_id="task_1", python_callable=my_func)
task_2 = BashOperator(
    task_id="task_2",
    bash_command="{{ ti.xcom_pull(task_ids='task_1') }}",  # AIR004
)

Suggested fix

task_2 = BashOperator(
    task_id="task_2",
    bash_command=task_1.output,
)

Template patterns detected

{{ ti.xcom_pull(task_ids='...') }} and {{ task_instance.xcom_pull(task_ids='...') }}
Positional argument: {{ ti.xcom_pull('...') }}
Both task_id= and task_ids= keyword forms
Various whitespace and quote styles

What it allows (no false positives)

Mixed content strings: "echo {{ ti.xcom_pull(task_ids='task_1') }}"
Non-default key arguments: "{{ ti.xcom_pull(task_ids='task_1', key='my_key') }}"
Already using .output: task_1.output
List of task_ids: "{{ ti.xcom_pull(task_ids=['a', 'b']) }}"
Non-operator/sensor calls (e.g., DAG(...))

Unsafe fix

When the referenced task_id matches a variable in scope (either an operator assignment or a @task-decorated function), an unsafe fix is provided that replaces the template string with <variable>.output. When no matching variable is found, the diagnostic is still reported but without an auto-fix.

Test Plan

Snapshot tests in AIR004.py covering:
- Violations with fixes: ti.xcom_pull, task_instance.xcom_pull, positional args, double quotes, no-space braces, singular task_id keyword, sensors, provider operators, and @task-decorated function sources.
- Violations without fixes: referencing a task_id not visible as a variable in the current scope.
- Non-violations: mixed content strings, extra keyword arguments, already using .output, regular strings, non-string arguments, list task_ids, and non-operator calls.
Unit tests for the parse_xcom_pull_template parser covering all template variants and rejection cases.
Clippy passes with no warnings.

related: apache/airflow#43176

astral-sh-bot · 2026-02-26T15:01:43Z

`ruff-ecosystem` results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+3 -0 violations, +0 -0 fixes in 1 projects; 55 projects unchanged)

apache/airflow (+3 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --no-fix --output-format concise --preview --select ALL

+ providers/amazon/tests/system/amazon/aws/example_dms_serverless.py:329:32: AIR201 Use the `.output` attribute on the task object for "create_replication_config" instead of `xcom_pull` in a template string
+ providers/google/tests/system/google/cloud/vertex_ai/example_vertex_ai_feature_store.py:174:32: AIR201 Use the `.output` attribute on the task object for "sync_task" instead of `xcom_pull` in a template string
+ providers/google/tests/system/google/cloud/vertex_ai/example_vertex_ai_feature_store.py:185:32: AIR201 Use the `.output` attribute on the task object for "sync_task" instead of `xcom_pull` in a template string

Changes by rule (1 rules affected)

code	total	+ violation	- violation	+ fix	- fix
AIR201	3	3	0	0	0

Related: astral-sh/ruff#23583

amyreese · 2026-02-27T02:48:51Z

cc @sjyangkevin for review?

sjyangkevin · 2026-02-27T02:53:14Z

cc @sjyangkevin for review?

would be great to also have @Lee-W to have a look when he is available.

Related: astral-sh/ruff#23583

Lee-W · 2026-03-05T02:13:47Z

Hey, thanks for creating this. I'm actaully good with adding this, but it would be better if we had a stronger consensus across the community. I'll start a discussion in the dev list today.

ntBre · 2026-03-05T16:53:04Z

Thanks @Lee-W and @sjyangkevin for taking a look! I think I'll convert this back to a draft for now until there's consensus on the Airflow side.

Dev-iL · 2026-03-23T11:31:01Z

@ntBre @amyreese @Lee-W @sjyangkevin

The Airflow community approved this rule, so I'm marking it "Ready for review".

Refs:

Lee-W

a few nits. but overall looks good!

Lee-W · 2026-03-24T07:12:22Z

+# Mixed content (not just xcom_pull)
+task_10 = BashOperator(
+    task_id="task_10",
+    bash_command="echo {{ ti.xcom_pull(task_ids='task_1') }}",


I guess we're worrying about false possitive?

Shouldn't we? What do you suggest?

I meant why shouldn't we replace if it's mixed content

In other cases, the template (string) is replaced by a different type (xcom). In a mixed context we need to keep a string, which relies on the correct serializability of xcoms. IMO this is more trouble than it's worth.

However, let's consider your proposal for a second - what should be the replacement in this example?

bash_command="echo {{ ti.xcom_pull(task_ids='task_1') }}", # original bash_command="echo {{ task_1.output }}", bash_command="echo " + task_1.output, bash_command=f"echo {task_1.output}", ...

I think it should be bash_command="echo {{ task_1.output }}",

sjyangkevin

Thanks! Overall looks good to me but one more small feedback. @Lee-W when you have time, would you mind also have a look see if the following valid?

Should we also handle an edge case that when the task is defined in a TaskGroup, let's say taskgroup_1. The callable task_id will be group_id.task_id, for example

task_4 = BashOperator(
    task_id="task_4",
    bash_command="{{ ti.xcom_pull('taskgroup_1.task_1') }}",
)

https://www.astronomer.io/docs/learn/task-groups#task_id-in-task-groups

MichaReiser · 2026-03-30T16:36:48Z

@Lee-W, how many rules do you plan on adding? I'm asking because adding and maintaining all these airflow rules is a considerable effort on our end. Don't get me wrong, the contributions are great, but it's unfortunately not at zero cost for us. If you plan on adding many rules, I think it's best if we discuss that first and how we can scale this process.

Dev-iL · 2026-03-30T17:03:31Z

If you plan on adding many rules, I think it's best if we discuss that first and how we can scale this process.

By all means - let's discuss!

We don't know how many of these rules there will be, but they will likely come one at a time in the future.
If ruff supported plugins, so that rules could be proposed/maintained/reviewed entirely on the Airflow side, that would probably be the most scalable solution long term.

potiuk · 2026-03-30T17:08:23Z

Yeah. I know we've heard in the past that plugin support is not really priority, but possibly we could do something about it now? We would actually love if Airflow rules were somewhere outside of main ruff core and if we could just configure it or even ask our users to install plugins separately. I am not sure if plugins is something on the roadmap now @MichaReiser ?

Lee-W · 2026-03-31T08:21:09Z

@MichaReiser, we don't have a clear understanding of whether we should add more rules at this time.

However, we definitely need to discuss our next steps regarding this matter!

Currently, if anyone has ideas about best practices for Airflow, they will need to initiate a discussion and build consensus within the Airflow community. This process takes time and doesn't happen often, so I don’t expect many changes.

Lee-W

LGTM

MichaReiser · 2026-04-08T09:46:07Z

Currently, if anyone has ideas about best practices for Airflow, they will need to initiate a discussion and build consensus within the Airflow community. This process takes time and doesn't happen often, so I don’t expect many changes.

Thanks, this sounds good to me.

Plugin support is on our mind. But we first want to push some highly demanded and larger user-facing features (warning severity, human-readable names, rule recategorization, unified format/check command). I suspect that plugins will be high up on our feature list once these are out.

MichaReiser

Thank you. This overall looks good. The Jinja parsing makes me a bit uneasy. I think there are a few more cases that need handling and there's potential for more code reuse.

MichaReiser · 2026-04-09T12:40:19Z

+    }
+
+    // Check keyword arguments for xcom_pull template strings.
+    for keyword in &*call.arguments.keywords {


Does this indeed apply to all keyword arguments or can we restrict the rule to a few known keyword names? What if the arguments are passed as positional arguments (or are they required to be keyword arguments)?

In Airflow, any operator argument can be a template field — it's determined by the operator's template_fields class attribute and varies per operator. Since the {{ ti.xcom_pull(...) }} pattern is specific enough to avoid false positives, I now check all arguments (both positional and keyword) with a code comment explaining the rationale. Positional template strings are rare in operator calls but technically possible.

Are there some known fields that we can always skip? E.g. task_id, dag and task_group?

Can you please elaborate or give an example of the use case you have in mind?

This rule was made to detect a common (and outdated) pattern where specifically the return value of a task is retrieved. The reason for the proliferation of the pattern is because it was the recommended (only?) way of doing things for a long time, and since this is what appeared in the docs - that's what users ended up doing. While it is possible to use positional args to call the function i.e.

ti.xcom_pull("task") # Plausible ti.xcom_pull("task", None) # Uncommon ti.xcom_pull("task", None, "return_value") # Uncommon

I think the above are so uncommon that false-negatives are not a concern in practice. Moreover, xcom_pull has 5 "positional-able" args (see below) - and we don't want to touch it if it the user provides anything except just task_ids and (optionally) key.

To conclude - I think that dealing with arbitrary templates is outside the scope of this rule.

CC: @Lee-W @sjyangkevin @potiuk

The current signature of xcom_pull:

def xcom_pull( self, task_ids: str | Iterable[str] | None = None, dag_id: str | None = None, key: str = XCOM_RETURN_KEY, # "return_value" include_prior_dates: bool = False, session: Session = NEW_SESSION, *, map_indexes: int | Iterable[int] | None = None, default: Any = None, run_id: str | None = None, ) -> Any:

MichaReiser · 2026-04-10T07:56:36Z

+    }
+
+    // Check keyword arguments for xcom_pull template strings.
+    for keyword in &*call.arguments.keywords {


Are there some known fields that we can always skip? E.g. task_id, dag and task_group?

- Rewrite parse_xcom_pull_template using ruff_python_trivia::Cursor instead of strip_prefix/strip_suffix chains - Extract reusable helpers: eat_whitespace, parse_identifier, parse_quoted_string - Add whitespace tolerance between all token pairs (e.g. ti . xcom_pull) - Bail on escaped quotes in string content - Broaden argument checking to cover both positional and keyword args with explanatory comment about template_fields - Add Fix safety doc section - Add unit tests for new whitespace/escape/unknown-keyword handling Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

…_whitespace - Use `iter_source_order` for argument iteration (MichaReiser) - Use `char::is_whitespace` instead of `is_ascii_whitespace` to match Jinja's Unicode whitespace semantics (MichaReiser) - Support reordered keyword arguments: `key='return_value', task_ids='...'` is now recognized in addition to the existing order (MichaReiser + Lee-W) - Extract `parse_task_id_value` helper for list/tuple wrapping logic - Add unit tests for reordered keyword patterns - Add fixture trigger case for reordered keywords (task_25) - Show modern `.output` replacement pattern as comment on mixed-content test - Regenerate snapshot in current insta format Co-Authored-By: Claude Opus 4.6 <[email protected]>

MichaReiser · 2026-04-13T14:52:41Z

This looks good to me, but Codex had two findings. I find it difficult to assess whether they're correct because I'm unfamiliar with airflow. Could someone take a look:

False positive on a non-templated field:
from airflow.operators.bash import BashOperator
from airflow.operators.python import PythonOperator

def f():
    pass

task_1 = PythonOperator(task_id="task_1", python_callable=f)

BashOperator(
    task_id="{{ ti.xcom_pull(task_ids='task_1') }}",  # currently flagged by AIR201
    bash_command="echo hi",
)
Why this is wrong: task_id is not a Jinja template_field, so Airflow will not render it. AIR201 should not inspect it.

Wrong autofix for a TaskFlow @task:
from airflow.decorators import task
from airflow.operators.bash import BashOperator

@task
def extract_data():
    return "value"

BashOperator(
    task_id="consumer",
    bash_command="{{ ti.xcom_pull(task_ids='extract_data') }}",
)
Current suggested fix in the PR:
BashOperator(
    task_id="consumer",
    bash_command=extract_data.output,
)
Why this is wrong: for TaskFlow tasks, you need the called task object / XComArg, e.g. extract_data() or a variable bound to that result, not the decorator object’s .output.

Dev-iL · 2026-04-13T15:07:06Z

False positive on a non-templated field:

Codex is technically correct here, but I don't think anyone does this. If they do, it's likely a mistake, and while it may be flagged for the wrong reason, it would indicate to the user that something's up. Also, not sure a template like that is even a valid task_id (there are some rules it needs to follow which I cannot find at the moment).

Wrong autofix for a TaskFlow @task:

This one's true, but the original syntax is already wrong for the exact same reason. We wouldn't be replacing working code with broken one.

MichaReiser · 2026-04-14T06:47:54Z

This is great work. Thank you

Dev-iL · 2026-04-14T06:49:59Z

This is great work. Thank you

Thank you for bringing this over the finish line!

astral-sh-bot Bot assigned amyreese Feb 26, 2026

ntBre added rule Implementing or modifying a lint rule preview Related to preview mode features labels Feb 26, 2026

Dev-iL added a commit to Dev-iL/airflow that referenced this pull request Feb 26, 2026

Fix AIR004 in some example DAGs

8542787

Related: astral-sh/ruff#23583

Dev-iL mentioned this pull request Feb 26, 2026

Fix AIR004* in multiple example DAGs apache/airflow#62529

Merged

1 task

Dev-iL added a commit to Dev-iL/airflow that referenced this pull request Feb 26, 2026

Fix AIR004 in some example DAGs

9b456ab

Related: astral-sh/ruff#23583

Dev-iL force-pushed the 2602/airflow/xcom_template branch from c0c2205 to fb18572 Compare February 26, 2026 17:20

Dev-iL added a commit to Dev-iL/airflow that referenced this pull request Feb 26, 2026

Fix AIR004 in some example DAGs

13deee0

Related: astral-sh/ruff#23583

sjyangkevin reviewed Feb 27, 2026

View reviewed changes

Comment thread crates/ruff_linter/src/rules/airflow/rules/xcom_pull_in_template_string.rs

Dev-iL added a commit to Dev-iL/airflow that referenced this pull request Feb 27, 2026

Fix AIR004 in some example DAGs

f4a9293

Related: astral-sh/ruff#23583

Dev-iL force-pushed the 2602/airflow/xcom_template branch 3 times, most recently from ca33d94 to 25ab2b5 Compare March 3, 2026 20:19

Dev-iL force-pushed the 2602/airflow/xcom_template branch from 25ab2b5 to 12d2489 Compare March 5, 2026 08:09

ntBre marked this pull request as draft March 5, 2026 16:53

ntBre mentioned this pull request Mar 5, 2026

[airflow] Implement task-branch-as-short-circuit (AIR004) #23579

Merged

Dev-iL force-pushed the 2602/airflow/xcom_template branch from 12d2489 to e51918c Compare March 20, 2026 11:15

Dev-iL marked this pull request as ready for review March 23, 2026 11:31

astral-sh-bot Bot requested a review from amyreese March 23, 2026 11:31

Lee-W reviewed Mar 24, 2026

View reviewed changes

Dev-iL force-pushed the 2602/airflow/xcom_template branch from e51918c to 64581b7 Compare March 24, 2026 13:55

Dev-iL changed the title ~~[airflow] Implement airflow-xcom-pull-in-template-string (AIR004)~~ [airflow] Implement airflow-xcom-pull-in-template-string (AIR201) Mar 24, 2026

Dev-iL force-pushed the 2602/airflow/xcom_template branch from 64581b7 to dcdeb6a Compare March 25, 2026 08:11

sjyangkevin reviewed Mar 29, 2026

View reviewed changes

Dev-iL mentioned this pull request Mar 29, 2026

Allow accessing a TaskGroup's members via [] apache/airflow#64430

Merged

1 task

Lee-W reviewed Apr 1, 2026

View reviewed changes

Dev-iL mentioned this pull request Apr 1, 2026

Explore and add static checks for DAGs for early detection of common issues apache/airflow#43176

Open

2 tasks

MichaReiser reviewed Apr 9, 2026

View reviewed changes

MichaReiser assigned MichaReiser and unassigned amyreese Apr 9, 2026

Dev-iL force-pushed the 2602/airflow/xcom_template branch from dcdeb6a to 2ab3e1b Compare April 10, 2026 06:52

MichaReiser reviewed Apr 10, 2026

View reviewed changes

Dev-iL and others added 4 commits April 10, 2026 14:27

Add AIR201: avoid unnecessary xcom_pull templates

752a511

Fix clippy doc_markdown lint for XCom

8ece240

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Dev-iL force-pushed the 2602/airflow/xcom_template branch from 1765da5 to 485dc03 Compare April 10, 2026 12:31

Dev-iL requested a review from MichaReiser April 12, 2026 09:32

Reduce code duplication for keyword argument handling

9859b49

MichaReiser merged commit 0921ed2 into astral-sh:main Apr 14, 2026
44 checks passed

Dev-iL deleted the 2602/airflow/xcom_template branch April 14, 2026 06:49

Dev-iL mentioned this pull request Apr 16, 2026

[airflow] Add mixed-content support to AIR201 #24673

Draft

BrewTestBot mentioned this pull request Apr 16, 2026

ruff 0.15.11 Homebrew/homebrew-core#277971

Merged

This was referenced Apr 27, 2026

Always include panic payload in panic diagnostic message #24873

Merged

ASCII identifier fast path #24876

Draft

Conversation

Dev-iL commented Feb 26, 2026

Summary

What the rule flags

Suggested fix

Template patterns detected

What it allows (no false positives)

Unsafe fix

Test Plan

Uh oh!

astral-sh-bot Bot commented Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ruff-ecosystem results

Linter (stable)

Linter (preview)

Uh oh!

amyreese commented Feb 27, 2026

Uh oh!

Uh oh!

sjyangkevin commented Feb 27, 2026

Uh oh!

Lee-W commented Mar 5, 2026

Uh oh!

ntBre commented Mar 5, 2026

Uh oh!

Dev-iL commented Mar 23, 2026

Uh oh!

Lee-W left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sjyangkevin left a comment

Choose a reason for hiding this comment

Uh oh!

MichaReiser commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Dev-iL commented Mar 30, 2026

Uh oh!

potiuk commented Mar 30, 2026

Uh oh!

Lee-W commented Mar 31, 2026

Uh oh!

Lee-W left a comment

Choose a reason for hiding this comment

Uh oh!

MichaReiser commented Apr 8, 2026

Uh oh!

MichaReiser left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Dev-iL Apr 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

astral-sh-bot Bot commented Feb 26, 2026 •

edited

Loading

`ruff-ecosystem` results

MichaReiser commented Mar 30, 2026 •

edited

Loading

Dev-iL Apr 10, 2026 •

edited

Loading

MichaReiser commented Apr 13, 2026 •

edited

Loading

Dev-iL commented Apr 13, 2026 •

edited

Loading