plugins: load plugins from providers#32692
Merged
Merged
Conversation
mobuchowski
force-pushed
the
load-plugins-from-providers
branch
from
July 19, 2023 13:06
2e0144b to
a126e8f
Compare
Member
|
Oh yeah. Good point. I briefly looked at it - looks like the right direction - I think there is a bit of a problem of where the plugins are loaded from during tests as some tests expect to load plugins from "example_dags/plugins" and I think this PR removes that path somehow, It's going to be much clearer and more straightforward after #32669 will be merged, because one of the things it does is that it unifies the unit test (local venv) / and CI environment for tesst so that it is configured in one place only and dynamically loaded via pytest fixture from here: So it might be a wise thing to wait after #32669 is merged, rebase and try to fix it there (happy to help/brainstorm - I got quite a bit more familiar with how unit test configuration is loaded by implementing the PRs) |
Member
BTW. We can't do it even for the future - because when Airflow gets providers installed as packages, using entrypoints is the only mechanism how it can discover plugins, there are no provider.yaml files any more in packaged version of Airflow (and can't be because providers are isolated and independent from Airflow and there might me different versions of both - community and 3rd-party providers installed together with Airflow, so the only way Airflow can find out what plugins are availabe is by querying the entrypoints. |
mobuchowski
force-pushed
the
load-plugins-from-providers
branch
2 times, most recently
from
July 24, 2023 10:35
b78c4ce to
e6bd521
Compare
uranusjr
reviewed
Jul 24, 2023
mobuchowski
force-pushed
the
load-plugins-from-providers
branch
3 times, most recently
from
July 24, 2023 19:28
1332b89 to
46f0ce1
Compare
Signed-off-by: Maciej Obuchowski <[email protected]>
mobuchowski
force-pushed
the
load-plugins-from-providers
branch
from
July 24, 2023 20:01
46f0ce1 to
1613bdb
Compare
mobuchowski
added
full tests needed
uranusjr
approved these changes
Jul 25, 2023
pgagnon
added a commit
to pgagnon/airflow
that referenced
this pull request
Feb 8, 2026
The __register_plugins function was using 'return' instead of 'continue' when encountering a duplicate plugin name. This caused the entire loop to exit early, preventing subsequent plugins from being loaded. Fixes a bug in apache#32692.
pgagnon
added a commit
to pgagnon/airflow
that referenced
this pull request
Feb 8, 2026
The __register_plugins function was using 'return' instead of 'continue' when encountering a duplicate plugin name. This caused the entire loop to exit early, preventing subsequent plugins from being loaded. Fixes a bug in apache#32692.
pgagnon
added a commit
to pgagnon/airflow
that referenced
this pull request
Feb 8, 2026
The __register_plugins function was using 'return' instead of 'continue' when encountering a duplicate plugin name. This caused the entire loop to exit early, preventing subsequent plugins from being loaded. Fixes a bug in apache#32692.
pgagnon
added a commit
to pgagnon/airflow
that referenced
this pull request
Feb 9, 2026
The __register_plugins function was using 'return' instead of 'continue' when encountering a duplicate plugin name. This caused the entire loop to exit early, preventing subsequent plugins from being loaded. Fixes a bug in apache#32692.
pgagnon
added a commit
to pgagnon/airflow
that referenced
this pull request
Feb 9, 2026
The __register_plugins function was using 'return' instead of 'continue' when encountering a duplicate plugin name. This caused the entire loop to exit early, preventing subsequent plugins from being loaded. This is a regression introduced in apache#32692.
potiuk
pushed a commit
that referenced
this pull request
Feb 11, 2026
The __register_plugins function was using 'return' instead of 'continue' when encountering a duplicate plugin name. This caused the entire loop to exit early, preventing subsequent plugins from being loaded. Fixes a bug in #32692.
ytoprakc
pushed a commit
to ytoprakc/airflow
that referenced
this pull request
Feb 12, 2026
) The __register_plugins function was using 'return' instead of 'continue' when encountering a duplicate plugin name. This caused the entire loop to exit early, preventing subsequent plugins from being loaded. Fixes a bug in apache#32692.
Ratasa143
pushed a commit
to Ratasa143/airflow
that referenced
this pull request
Feb 15, 2026
) The __register_plugins function was using 'return' instead of 'continue' when encountering a duplicate plugin name. This caused the entire loop to exit early, preventing subsequent plugins from being loaded. Fixes a bug in apache#32692.
choo121600
pushed a commit
to choo121600/airflow
that referenced
this pull request
Feb 22, 2026
) The __register_plugins function was using 'return' instead of 'continue' when encountering a duplicate plugin name. This caused the entire loop to exit early, preventing subsequent plugins from being loaded. Fixes a bug in apache#32692.
Subham-KRLX
pushed a commit
to Subham-KRLX/airflow
that referenced
this pull request
Mar 4, 2026
) The __register_plugins function was using 'return' instead of 'continue' when encountering a duplicate plugin name. This caused the entire loop to exit early, preventing subsequent plugins from being loaded. Fixes a bug in apache#32692.
dominikhei
pushed a commit
to dominikhei/airflow
that referenced
this pull request
Mar 11, 2026
) The __register_plugins function was using 'return' instead of 'continue' when encountering a duplicate plugin name. This caused the entire loop to exit early, preventing subsequent plugins from being loaded. Fixes a bug in apache#32692.
Ankurdeewan
pushed a commit
to Ankurdeewan/airflow
that referenced
this pull request
Mar 15, 2026
) The __register_plugins function was using 'return' instead of 'continue' when encountering a duplicate plugin name. This caused the entire loop to exit early, preventing subsequent plugins from being loaded. Fixes a bug in apache#32692.
radhwene
pushed a commit
to radhwene/airflow
that referenced
this pull request
Mar 21, 2026
) The __register_plugins function was using 'return' instead of 'continue' when encountering a duplicate plugin name. This caused the entire loop to exit early, preventing subsequent plugins from being loaded. Fixes a bug in apache#32692.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In some cases - like when provider is included by
breeze prod-image build --install-providers-from-sourcesplugins are not loaded, as they are not separate package with entrypoint then. This PR adds loading plugins fromprovider.yml. I think this is somehow similar to what @potiuk is doing with stuff like config or executors.In some cases this would make entrypoint plugins double-load - so we check if the plugin is already loaded. We could avoid that by not generating entrypoints for plugins, but this would make them not load on earlier versions of Airflow.