Suppress import errors for providers from sources #22579
Conversation
When we are running airflow locally with providers installed from sources, often many providers will be discovered which we haven't installed the deps for. This generally results in a very large amount of traceback logging, which has a very negative effect on usefulness of terminal output. Here we suppress this error logging for providers that are installed from sources.
potiuk
left a comment
potiuk
left a comment
There was a problem hiding this comment.
Looks cool! Thanks!
NIT: I'd use Enum or bool for providers/sources, but having string is nice for debugging too, so no problem wiht it (and TIL about post_init_)
github-actions
bot
added
the
okay to merge
|
The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease. |
|
Also 'airflow providers list' cmd needs fixing too :) |
|
(Remnant from time when ProviderInfo was just a tuple it seems). |
yeah thought about Enum but thought it was too heavy handed for this But bool is a good idea then we don't need post_init |
PR apache#22579 suppressed provider import log warnings when running from sources. For whatever reason, CI didn't run test_providers_manager.py so I didn't catch that changes were needed there.
PR #22579 suppressed provider import log warnings when running from sources. For whatever reason, CI didn't run test_providers_manager.py so I didn't catch that changes were needed there.
ephraimbuddy
added
the
changelog:skip
4 participants
When we are running airflow locally with providers installed from sources, often many providers will be discovered which we haven't installed the deps for. This generally results in a very large amount of traceback logging, which has a very negative effect on usefulness of terminal output. Here we suppress this error logging for providers that are installed from sources.