Skip to content

Comments

[airflow] Add ruff rules to catch deprecated attribute access from context key for Airflow 3.0 (AIR301)#22850

Merged
ntBre merged 5 commits intoastral-sh:mainfrom
sjyangkevin:air301-context-key-change
Feb 9, 2026
Merged

[airflow] Add ruff rules to catch deprecated attribute access from context key for Airflow 3.0 (AIR301)#22850
ntBre merged 5 commits intoastral-sh:mainfrom
sjyangkevin:air301-context-key-change

Conversation

@sjyangkevin
Copy link
Contributor

@sjyangkevin sjyangkevin commented Jan 25, 2026

Summary

Context:
apache/airflow#41641

  1. Replace external_trigger check with DagRunType apache/airflow#45961
  • create_dagrun removed from airflow...DAG (This has already been implemented in AIR301)
  • context key dag_run.external_trigger removed
  1. feat(task_sdk): add support for inlet_events in Task Context apache/airflow#45960
  • context["inlet_events"]["url"] → context["inlet_events"][Asset("url")]
  1. Rename dataset related python variable names to asset apache/airflow#41348
  • context key triggering_dataset_events → triggering_asset_events

The existing AIR301 rules can detect when users access a removed key such as execution_date through Airflow's context. For example, context["execution_date"], or context["dag_run"]. However, if an attribute is deprecated from a context key, such as context["dag_run"].external_trigger, the current implement will not flag it.

This PR adds the logic for such check, and add two rules to flag the deprecated attribute for the "dag_run" and "inlet_events" context key. In addition to this, "triggering_dataset_events" is a deprecated context key which can be handled by the existing rule. However, the existing rule doesn't raise a diagnostic. Hence, the rule logic is refactored a little bit, such that we can add this check and suggest a Replacement::Rename.

Test Plan

The test cases have been added to AIR301_context.py, and all the tests have been run locally and success. @Lee-W , could you please review it when you have time, thanks!

Notes

In #22376, we introduced some improvements to the AIR301 code. I will re-base this PR when we are all good on that, so it can pick up those code structure improvements, and updated rules.

@astral-sh-bot
Copy link

astral-sh-bot bot commented Jan 25, 2026

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

@sjyangkevin sjyangkevin marked this pull request as ready for review January 25, 2026 21:54
@sjyangkevin sjyangkevin marked this pull request as draft January 25, 2026 21:54
*range,
);
diagnostic.set_fix(Fix::safe_edit(Edit::range_replacement(
"\"triggering_asset_events\"".to_string(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will this also check context['triggering_asset_events']?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we have two functions here and they are called in different places.
1.) check_context_key_usage_in_call on Expr::Call(call_expr @ ExprCall {func, arguments, ..},)
2.) check_context_key_usage_in_subscript on Expr::Subscript(subscript_expr)

The check_context_key_usage_in_subscript will capture context['triggering_dataset_events'] and raise diagnostic to suggest update to use context['triggering_asset_events']

Screenshot from 2026-01-27 22-50-51

)
}
"uri" if is_removed_context_key_attribute(checker, attr_expr, "inlet_events", attr) => {
Replacement::AttrName("asset.uri")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry... my description was totally wrong...


Here's what we want to check

# airflow 2.11

context["inlet_events"]["this://is-url"] # this is not ok in airflow 3+
context["inlet_events"][Dataset("this://is-url")] # this is still ok

We do not support accessing inlet events through a string (used to be the URI of Dataset) now. We can either change it to Asset(uri="this://is-url") or just warn the user to use the Asset object instead of a string.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah I see. In this case, instead of checking the uri attribute on context["inlet_events"], we actually should check if context["inlet_events"] is being accessed again through subscript, and the key is of type str. If we detect this, we should suggest to change the str (i.e., the url) to Asset(uri="this://is-url")

also, I wanted to double-check that the implementation for external_trigger is still correct.

@ntBre ntBre added the rule Implementing or modifying a lint rule label Jan 28, 2026
@sjyangkevin sjyangkevin force-pushed the air301-context-key-change branch 2 times, most recently from 3750166 to e37e963 Compare February 3, 2026 03:47
Copy link
Contributor Author

@sjyangkevin sjyangkevin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Lee-W , I've made the following updates to this file. Could you please have a look and let me know if the rules are implemented properly?

Copy link
Contributor Author

@sjyangkevin sjyangkevin Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new rule added here are:

  1. triggering_dataset_events -> triggering_asset_events, use Rename to suggest a fix.
  2. context["dag_run"].external_trigger is deprecated, use Message to warn.
  3. context["inlet_events"]["this://is-url"] is deprecated. use Message to warn, provide a example to use Asset("this://is-url").
  1. and 3. above support detection for .get() as well, or even variable binding inlet_events = context["inlet_events"]; inlet_events["this://is-url"]

How the rules for the context keys are triggered.

  1. check_context_key_usage_in_call is used to capture case context.get("key"), triggered on Expr::Call
  2. check_context_key_usage_in_subscript is used to capture case context["key"], triggered on Expr::Subscript

In the current implementation, we need to implement the same rule in two places mentioned above. Therefore, I've done a refactor to extract the rules and the report diagnostic into the following functions.

  1. check_deprecated_context_key (check for deprecated context key, e.g., context["triggering_dataset_events"])
  2. check_deprecated_context_key_value_access (check for deprecated access on context key's value, e.g., context["inlet_events"]["this://is-url"])
  3. check_removed_attribute_access_on_context_key (check for deprecated access on attribute on context key's value, e.g., context["dag_run"].external_trigger)

In this way, we can keep our rule and diagnostic reporting in a single place, but calling these functions in the two rule trigger functions above.

I also extracted the common check logic into multiple helper functions.

  1. is_context_variable: checks if an expression is a valid Airflow context variable. If should be either an kwarg parameter **context, or a variable from get_current_context().
  2. is_context_key_access: checks for key access through subscription context["key"] or a call context.get().
  3. is_value_from_context_key: checks if an expression is or resolves to an access of a specific context key. e.g., inlet_events = context["inlet_events"]; inlet_events.
  4. is_removed_context_key_attribute: checks if an attribute access on a specific context key; e.g., dagrun = context["dag_run"]; dagrun.external_trigger.

@sjyangkevin sjyangkevin requested a review from Lee-W February 3, 2026 04:16
@sjyangkevin sjyangkevin force-pushed the air301-context-key-change branch from e37e963 to e12a417 Compare February 4, 2026 21:35
@sjyangkevin sjyangkevin marked this pull request as ready for review February 4, 2026 21:46
@sjyangkevin
Copy link
Contributor Author

Hi @Lee-W , @ntBre , sorry for constantly pinging you. I just re-base this PR after the merge of #22376, this should be ready for review too. Please let me know if there is any feedback. Thanks!

@ntBre
Copy link
Contributor

ntBre commented Feb 4, 2026

No worries about the pings, happy to take a look!

@ntBre ntBre requested review from Lee-W and ntBre and removed request for Lee-W February 4, 2026 21:54
Copy link
Contributor

@Lee-W Lee-W left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the error message was wrong, but everything else looks great!

Airflow3Removal {
deprecated: "inlet_events[\"<uri>\"]".to_string(),
replacement: Replacement::Message(
"Accessing `inlet_events` via a string key is deprecated; use `inlet_events[Asset(\"<uri>\")]` instead (e.g., context[\"inlet_events\"][Asset(\"this://is-url\")]).",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"Accessing `inlet_events` via a string key is deprecated; use `inlet_events[Asset(\"<uri>\")]` instead (e.g., context[\"inlet_events\"][Asset(\"this://is-url\")]).",
"Accessing `inlet_events` via a string key is deprecated; use `inlet_events[Asset(uri=\"<uri>\")]` instead (e.g., context[\"inlet_events\"][\"this://is-url\"]).",

@sjyangkevin sjyangkevin force-pushed the air301-context-key-change branch from e12a417 to 9c997ea Compare February 6, 2026 19:21
| ^^^^^^^^^^^^^^^
198 | print(inlet_events.get("this://is-url"))
|
help: Accessing `inlet_events` via a string key is deprecated; use `context["inlet_events"][Asset(uri="this://is-url")]` instead of `context["inlet_events"]["this://is-url"]`.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Lee-W , thank you for caching this!! I just made an update to the message to make it bit more accurate and consistent. Can you have a look again? Thanks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great :)

Copy link
Contributor

@ntBre ntBre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thank you!

Copy link
Contributor

@Lee-W Lee-W left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks!

@ntBre ntBre merged commit 5338823 into astral-sh:main Feb 9, 2026
42 checks passed
carljm added a commit to Hugo-Polloli/ruff that referenced this pull request Feb 9, 2026
* main: (45 commits)
  [ty] Fix wrong inlay hints for overloaded function arguments (astral-sh#23179)
  [ty] Respect `@no_type_check` when combined with other decorators (astral-sh#23177)
  [ty] Use type context when inferring constructor argument types (astral-sh#23139)
  [`airflow`] Add ruff rules to catch deprecated attribute access from context key for Airflow 3.0 (`AIR301`) (astral-sh#22850)
  Support formatting `pycon` markdown code blocks (astral-sh#23112)
  Markdown formatting in LSP (astral-sh#23063)
  Instruct Claude to use comments more sparingly (astral-sh#23181)
  [`flake8-gettext`] Fix false negatives for plural argument of ngettext (`INT001`, `INT002`, `INT003`) (astral-sh#21078)
  [ty] Invoking goto-def on parentheses of a class constructor call takes you too constructor method
  [ty] Make goto definition on class constructor always go to class definition
  [ty] Assign lower completions ranking to deprecated functions and classes (astral-sh#23089)
  [ty] Fix parameter references across files via keyword args (astral-sh#23012)
  [ty] Exclude enclosing class for base completions (astral-sh#23141)
  [`pyupgrade`] Fix syntax error on string with newline escape and comment (`UP037`) (astral-sh#22968)
  [ty] Improve documentation for `expect_single_definition` method (astral-sh#23175)
  [ty] Configure check mode for all projects
  Add `home-assistant` to ecosystem projects (astral-sh#23132)
  Add tabbed shell completion documentation (astral-sh#23169)
  Bump typing conformance-suite pin (astral-sh#23174)
  [ty] Fix invalid diagnostic location for a sub-call to a specialized ParamSpec (astral-sh#23036)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

rule Implementing or modifying a lint rule

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants