Provider packages are installed by default in production image #12154
Conversation
potiuk
force-pushed
the
fix-missing-provider-packages-in-production-docker-image
branch
2 times, most recently
from
11a7a90 to
6ac15dc
Compare
potiuk
added
the
priority:critical
potiuk
force-pushed
the
fix-missing-provider-packages-in-production-docker-image
branch
from
6ac15dc to
7c2d88a
Compare
Yes. Correct. it should only happen when "install_providers_from_sources" is set. that is the right fix |
This is a fix to a problem introduced in apache#10806. The change turned provider packages into namespace packages - which made them ignored by find_packages function from setup tools - thus prodiuction image build automatically and used by Kubernetes tests did not have the provider packages installed. This PR fixes it and adds future protection during CI tests of production image to make sure that provider packages are actually installed. Fixes apache#12150
potiuk
force-pushed
the
fix-missing-provider-packages-in-production-docker-image
branch
from
7c2d88a to
096d5ac
Compare
|
Should be fixed now. As discussed - later when we will release provider packages we will switch to the mode of building the package using wheels. But I do not think we need to do it now, I think we need to release PyPI packages first and then we will synchronize the process. I yet have to see how we are going to build those. The Docker File and scripts have all the features needed to build the "master" image differently - using packages. However requiring going through the hoops of building all packages and installing them locally for local development and testing is not a good idea, so building from sources including all providers should remain to help developers who are actually working on improving the prod image. |
potiuk
deleted the
fix-missing-provider-packages-in-production-docker-image
branch
3 participants
This is a fix to a problem introduced in #10806. The change
turned provider packages into namespace packages - which made
them ignored by find_packages function from setup tools - thus
prodiuction image build automatically and used by Kubernetes
tests did not have the provider packages installed.
This PR fixes it and adds future protection during CI tests of
production image to make sure that provider packages are
actually installed.
Fixes #12150
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.