-
Notifications
You must be signed in to change notification settings - Fork 16.3k
Description
Apache Airflow version: 1.10.14
Environment: Python 3.8
- Install tools: pipenv
What happened:
So I appear to have stumbled across a version incompatibility for airflow-1.10.14
apache-airflow[aws] was modified with this pr to restrict boto to boto3~=1.10,<1.11 with the comment for snowflake
When attempting to install the latest amazon-provider backport apache-airflow-backport-providers-amazon==2020.11.23->-r I’m faced with an incompatible version in resolved dependancies as the backport requires boto3<2.0.0,>=1.12.0
attempting to install aws extra and amazon-provide backport results in a version incompatibility error.
There are incompatible versions in the resolved dependencies:
boto3<1.11,~=1.10 (from apache-airflow[aws]==1.10.14->-r /var/folders/dw/8s4dltks7bg5ws8kht1wfbqw0000gp/T/pipenvtfsk4ltbrequirements/pipenv-ukucfwxw-constraints.txt (line 2))
boto3<2.0.0,>=1.12.0 (from apache-airflow-backport-providers-amazon==2020.11.23->-r /var/folders/dw/8s4dltks7bg5ws8kht1wfbqw0000gp/T/pipenvtfsk4ltbrequirements/pipenv-ukucfwxw-constraints.txt (line 3))
What you expected to happen:
Airflow has compatible versions.
How to reproduce it:
1. Create Pipfile with the following contents
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true
[dev-packages]
[packages]
apache-airflow = {extras = ["aws"],version = "*"}
# https://github.com/apache/airflow/blob/master/README.md#backport-packages
apache-airflow-backport-providers-amazon = "*"
[requires]
python_version = "3.8"
Anything else we need to know:
Suggested solution: move all snowflake version modifications out of related airflow extras and directly into the snowflake extra.
Slack Thread: https://apache-airflow.slack.com/archives/CCQ7EGB1P/p1608057981159400