Skip to content

Conversation

@potiuk
Copy link
Member

@potiuk potiuk commented Mar 12, 2021

Sometimes base python image patchlevel might case failure of tests.
This happens for example with test_views.py tests fixed in #14719
where CVE fix in all python versions caused our tests to fail.

There was an error in our scripts - when --force-pull-images
were used, the base python version was not updated to the
latest version even if there was a newer one and it caused our
images to bounce few times between two latest patchlevels
when they were manually refreshed.

This change fixes it so that the base python image is always used
when

a) FORCE_PULL_IMAGES is true or
b) UPGRADE_TO_NEWER_DEPENDENCIES is != false

This will cause python upgrade in two cases:

  • when images are rebuilt with --force-pull-images locally
  • when images are upgraded with newer dependencies on master

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

@potiuk potiuk requested review from ashb and kaxil as code owners March 12, 2021 00:45
@potiuk potiuk requested review from mik-laj and turbaszek March 12, 2021 00:45
@potiuk potiuk force-pushed the fix-force-pulling-base-python-image branch from 2333caf to 5fc06b2 Compare March 13, 2021 12:54
Sometimes base python image patchlevel might case failure of tests.
This happens for example with test_views.py tests fixed in apache#14719
where CVE fix in all python versions caused our tests to fail.

There was an error in our scripts - when --force-pull-images
were used, the base python version was not updated to the
latest version even if there was a newer one and it caused our
images to bounce few times between two latest patchlevels
when they were manually refreshed.

This change fixes it so that the base python image is always used
when

a) FORCE_PULL_IMAGES is true or
b) UPGRADE_TO_NEWER_DEPENDENCIES is != false

This will cause python upgrade in two cases:
 - when images are rebuilt with --force-pull-images locally
 - when images are upgraded with newer dependencies on master
@potiuk potiuk force-pushed the fix-force-pulling-base-python-image branch from 5fc06b2 to beedf45 Compare March 13, 2021 13:05
@potiuk
Copy link
Member Author

potiuk commented Mar 13, 2021

Hey @kaxil -> that should permanently fix the similar problems that we had with the python CVE fix recently

@github-actions github-actions bot added the full tests needed We need to run full set of tests for this PR to merge label Mar 13, 2021
@github-actions
Copy link

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest master at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

@potiuk potiuk merged commit 61b4482 into apache:master Mar 13, 2021
@potiuk potiuk deleted the fix-force-pulling-base-python-image branch March 13, 2021 18:38
potiuk added a commit that referenced this pull request Mar 23, 2021
Sometimes base python image patchlevel might case failure of tests.
This happens for example with test_views.py tests fixed in #14719
where CVE fix in all python versions caused our tests to fail.

There was an error in our scripts - when --force-pull-images
were used, the base python version was not updated to the
latest version even if there was a newer one and it caused our
images to bounce few times between two latest patchlevels
when they were manually refreshed.

This change fixes it so that the base python image is always used
when

a) FORCE_PULL_IMAGES is true or
b) UPGRADE_TO_NEWER_DEPENDENCIES is != false

This will cause python upgrade in two cases:
 - when images are rebuilt with --force-pull-images locally
 - when images are upgraded with newer dependencies on master

(cherry picked from commit 61b4482)
ashb pushed a commit that referenced this pull request Apr 15, 2021
Sometimes base python image patchlevel might case failure of tests.
This happens for example with test_views.py tests fixed in #14719
where CVE fix in all python versions caused our tests to fail.

There was an error in our scripts - when --force-pull-images
were used, the base python version was not updated to the
latest version even if there was a newer one and it caused our
images to bounce few times between two latest patchlevels
when they were manually refreshed.

This change fixes it so that the base python image is always used
when

a) FORCE_PULL_IMAGES is true or
b) UPGRADE_TO_NEWER_DEPENDENCIES is != false

This will cause python upgrade in two cases:
 - when images are rebuilt with --force-pull-images locally
 - when images are upgraded with newer dependencies on master

(cherry picked from commit 61b4482)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:dev-tools full tests needed We need to run full set of tests for this PR to merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants