Old libraries in setup.py causing dependency resolution to pull old transitive constraints (3 years+) #12120
Description
Dear and Wonderful Citizens,
I started to look at what libraries we have defined in the constraints-*.txt file and I am a bit surprised because we have this constraints defined on very old libraries.
https://github.com/apache/airflow/blob/053afe7/constraints-3.8.txt
Update (@potiuk): -> Just for clarity: constraints are automatically generated from setup.py so this is a matter of dependencies defined there. If we are to fix it, we will have to upgrade dependencies defined in setup.py NOT the constraints themselves.
Sometimes we have defined libraries that are over 3 years old, which can cause security problems. Old versions of the library may have vulnerabilities that have probably been fixed in newer versions.
I am most concerned about dependency conflicts. Old libraries are only compatible with old libraries, which can cause problems if the user wants to use a new version of the same library.
I think it's worth investigating where these limitations come from and why we can't use newer versions of these libraries.
You can see the list of libraries that need updating in the Jupyter interactive notebook.
https://colab.research.google.com/drive/1F5Lw8qNcxCvWaYUrGZ1x3W3v3080Dq0U#scrollTo=AfIBqzjo8UId