feat(AIP-76): support forward fan-out via forward kwarg on Window#67475
Merged
Conversation
boring-cyborg
Bot
added
area:API
Lee-W
force-pushed
the
partition-fanout-direction
branch
from
81572b9 to
eeda6f0
Compare
Lee-W
changed the title
Lee-W
force-pushed
the
partition-fanout-direction
branch
11 times, most recently
from
796a9ce to
89db186
Compare
phanikumv
reviewed
Jun 3, 2026
Lee-W
force-pushed
the
partition-fanout-direction
branch
from
89db186 to
c2e97e1
Compare
Lee-W
force-pushed
the
partition-fanout-direction
branch
from
c2e97e1 to
08fb555
Compare
Lee-W
force-pushed
the
partition-fanout-direction
branch
3 times, most recently
from
f389552 to
396d02c
Compare
uranusjr
reviewed
Jun 9, 2026
uranusjr
reviewed
Jun 9, 2026
uranusjr
reviewed
Jun 9, 2026
uranusjr
reviewed
Jun 9, 2026
Lee-W
force-pushed
the
partition-fanout-direction
branch
from
396d02c to
2174381
Compare
Lee-W
requested review from
bugraoz93,
gopidesupavan,
jason810496,
jscheffl and
potiuk
as code owners
Lee-W
force-pushed
the
partition-fanout-direction
branch
4 times, most recently
from
ee76e57 to
00a6427
Compare
Lee-W
force-pushed
the
partition-fanout-direction
branch
from
00a6427 to
3d5755a
Compare
WindowDirection.FORWARD now yields the trailing period ending at the upstream key (the half-open (anchor - period, anchor], mirror of BACKWARD) rather than the following period. The anchor is always the last member of the forward sequence. - WeekWindow(direction=FORWARD) on a Monday fans out the seven days ending at that Monday, not the next calendar week. - MonthWindow forward enumerates the trailing month ending at the anchor and is intentionally not calendar-month aligned, matching the week behaviour. Closes apache#65761.
Coerce the `direction` argument through `WindowDirection(...)` in `Window.__init__` so an invalid value raises `ValueError` at construction instead of silently behaving as FORWARD when the later `is` identity check fails to match a stray string. Applies to both the core and task-sdk copies.
Collapse the per-class `if direction is BACKWARD: shift; forward-generate` duplication in the temporal Window classes into a single `_directional_steps` helper, so the trailing-period rule lives in one place. Hour/Day/Week/Quarter/ Year share it via a step callable; MonthWindow stays bespoke (variable member count, open-closed backward interval).
… hook Replace the manual "keep in sync" header comment on the Task SDK Window copy with a `check-window-in-sync` prek hook that AST-compares the shared surface (class names, WindowDirection enum, direction default, serialize/deserialize bodies, expected_decoded_type) against the core copy and fails on drift.
Replace the top-level `WindowDirection` enum with a nested `Window.Direction`
so authors reference `Window.Direction.FORWARD` and import only `Window`.
Removes the standalone export from the SDK and core partition_mappers surfaces;
serialized values ("forward"/"backward") are unchanged. The core/SDK sync prek
hook is updated to track the nested name.
Lee-W
force-pushed
the
partition-fanout-direction
branch
from
3d5755a to
8b9f6f7
Compare
uranusjr
approved these changes
Jun 10, 2026
github-project-automation
Bot
moved this from In Review
to Done
in AIP-76 Asset Partitioning
Contributor
Backport failed to create: v3-2-test. View the failure log Run detailsNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
You can attempt to backport this manually by running: cherry_picker a8c9910 v3-2-testThis should apply the commit to the v3-2-test branch and leave the commit in conflict state marking After you have resolved the conflicts, you can continue the backport process by running: cherry_picker --continueIf you don't have cherry-picker installed, see the installation guide. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Fan-out today always produces keys inside the upstream window — weekly W10 (Mar 4–10) → days Mar 4–10. But the seven days that follow (Mar 11–17, i.e. W11) is just as valid a pattern. #65761 asks for a way to express that direction.
What
Adds
forward: bool = FalseonWindow. Passforward=Trueto fan out the next period instead:FanOutMapper is unchanged; the shift lives on the window because the math is per-window-type. Default False preserves existing behavior, and deserialize(... data.get("forward", False)) handles already-persisted Dags. For Month / Quarter / Year, the existing day-1 guard runs before the shift, so non-day-1 input with forward=True raises symmetrically with current behavior.
Test plan
TestForwardWindowcovers all six built-in windows (Hour / Day / Week / Month / Quarter / Year): forward shift produces the next period, day-1 guard fires before shift, serialize round-trip, and back-compat when the forward key is missing.Closes #65761.
Was generative AI tooling used to co-author this PR?
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.