Allow custom partition Window subclasses via plugin registry#68717
Conversation
Lee-W
requested review from
amoghrajesh,
ashb,
bolkedebruin and
potiuk
as code owners
FrankYang0529
left a comment
FrankYang0529
left a comment
There was a problem hiding this comment.
LGTM. One minor suggestion: do we want to add a custom window example to example_dags/plugins/?
Lee-W
force-pushed
the
custom-window-plugin-registry
branch
from
870a987 to
51f854c
Compare
|
sounds like a good idea! I didn't even know we have that. Thanks! |
phanikumv
left a comment
phanikumv
left a comment
There was a problem hiding this comment.
CI failing for the below reasons:-
In airflow-core/tests/unit/plugins/test_plugins_manager.py#L82
The new business_day_window.py example plugin makes this 11, not 10 — this is what's failing CI (assert 11 == 10).
assert len(plugins) == 11
In airflow-core/tests/unit/plugins/test_plugins_manager.py#L105
Same cause here, now four examples load, not three (assert 4 == 3). Bump the count and the comment.
assert len(plugins) == 4 # four are loaded from examples
Lee-W
force-pushed
the
custom-window-plugin-registry
branch
2 times, most recently
from
cb62c2d to
457fe26
Compare
Custom ``Window`` subclasses can now be registered on ``AirflowPlugin.windows``, serialized by import path, and run scheduler-side — mirroring custom timetables and partition mappers. This lets users implement calendar-aware rollup completeness (business days, holidays) as a user-land window instead of waiting on raw clock coverage. Unregistered non-core import paths are still rejected with ``WindowNotSupported``; the deserializer never imports an unregistered path. Newsfragment pending PR number.
Demonstrate registering a calendar-aware Window subclass via AirflowPlugin.windows, mirroring the existing custom priority-weight strategy example.
Lee-W
force-pushed
the
custom-window-plugin-registry
branch
from
457fe26 to
01c35aa
Compare
thanks, just fixed |
Backport successfully created: v3-3-testNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
|
…try (#68717) (#68836) Co-authored-by: Wei Lee <[email protected]>
4 participants
Why
Today custom
Windowsubclasses are rejected at serialization withWindowNotSupported— only the 7 builtins work.closes: #65756
What
A custom-window plugin registry, mirroring the existing custom-timetable / partition_mapper ones:
windowslist attribute onAirflowPlugin(windows = [MyWindow]).@cachedget_windows_plugins()collector.find_registered_custom_window()inserialization/helpers.py.encode_window/decode_windownow look non-core paths up in the registry (builtin fast-path unchanged); unregistered paths still raiseWindowNotSupportedand are never handed toimport_string.Was generative AI tooling used to co-author this PR?
Generated-by: [Claude] following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.