Apache Airflow Provider(s)

databricks

Versions of Apache Airflow Providers

apache-airflow-providers-databricks==2.7.0

Apache Airflow version

2.2.2

Operating System

Debian GNU/Linux 11 (bullseye)

Deployment

Other

Deployment details

No response

What happened

There was added import statement from the requests library to airflow/providers/databricks/hooks/databricks_base.py by this MR.

But minimal version of requests wasn't bumped up to >=2.27.0, as we can see it's still >=2.26.0, but that JSONDecodeError class added to requests library starting from 2.27.0 (see release notes).

It brings inconsistency between libraries and leads to following error:

  File "/usr/local/lib/python3.7/site-packages/airflow/providers/databricks/hooks/databricks.py", line 33, in <module>
    from airflow.providers.databricks.hooks.databricks_base import BaseDatabricksHook
  File "/usr/local/lib/python3.7/site-packages/airflow/providers/databricks/hooks/databricks_base.py", line 34, in <module>
    from requests.exceptions import JSONDecodeError
ImportError: cannot import name 'JSONDecodeError' from 'requests.exceptions' 

What you think should happen instead

it breaks DAGs

How to reproduce

Use any Databricks operator with requests==2.26.0, which is defined is minimal compatible version

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct