Limit pandas to <2.2#37748
Conversation
Taragolis
requested review from
hussein-awala,
jedcunningham,
potiuk,
uranusjr and
vincbeck
boring-cyborg
Bot
added
area:providers
provider:apache-hive
provider:common-sql
provider:exasol
provider:google
|
I'm not sure should we also rebuild images and regenerate constraints for 2.8.2 ? ❯ docker run -it --rm apache/airflow:2.8.2 bash
airflow@720faeb56401:/opt/airflow$ pip list | grep pandas
pandas 2.2.1
pandas-gbq 0.21.0 |
Yes. |
|
I will do i t - pandas is too important to get it broken |
|
Constraints updated, images are being rebuilt in https://github.com/apache/airflow/actions/runs/8068053366 |
|
Images are rebuilt (verified it). Announced at dev@ and users@, also marked it for 2.8.3 |
|
BTW, @Taragolis - I wonder if we could add a test to detect this issue? I think it might be good to add it for the future - this will also give us more confidence whe they will fix it and we will have to upgrade. WDYT ? |
|
@potiuk yeah we could have some integration tests for that, e.g. for MySQL and Postgres. Let me create a task for that, just for avoid situation that we discuss and finally forget to do that 🤣 |
ephraimbuddy
added
the
type:misc/internal
5 participants
closes: #37742
In pandas 2.2 minimal version of the sqlalchemy is 2.0, see: https://pandas.pydata.org/docs/whatsnew/v2.2.0.html#increased-minimum-versions-for-dependencies
However Airflow not fully supports it yet: #28723 even if could technically change the limitation in FAB also limit sqlalchemy to < 2.0, so the better solution that we have for now it is limit pandas in core extra and providers
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.