You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Bug]: gprcio limitation to < 1.66 in Python is problematic #34081
Some of the dependencies of Airflow (ydb) started to expect grpcio > 1.68.1 and apache-beam has < 1.66 - this is causing us to limit the other packages from installing - which we do not really like to do. It would be great to lift the upper-bound limitation if possible.
Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
There are at least two issue with new versions of GRPC:
Our Go tests were failing on grpc 1.66.0 and newer
Using grpcio==1.68.0, and some newer versions (haven't checked the latest version), causes pipeline failures due to ping timeout errors: SDK->Runner communication channel sometimes breaks down when the pipeline is running large volumes of data over the network (I have a repro that involves writing ~50GB of dummy data to GCS).
We are investigating this issues in close contact with GRPC team but would not recommend updating GRPC until this is rootcaused.
@damccorm@Amar3tto let's try to have this fixed in 2.72.0, given that we are only waiting for the cherrypick release for grpc/grpc#37710 . Let's work with @sergiitk 's team if we must delay the release for a couple of days until grpc's release is available and tested.
I spoke with @sergiitk offline, the team expects to start the release in the next 24 hrs, and it might take 1-2 days to finish. @sergiitk feel free to correct. thanks.
Thanks for the info! Really looking forward to it. And also from the testing perspective - airflow has quite a comprehensive overall testing harness on various levels, and we will be absolutely happy to kick it in and run the full suite of our tests - from complex cross-dependency resolution of things to actual operator tests that can use release candidates when ready - we can make PRs with updated dependencies to rc candidates and run full test suite very easily, so I am super happy to get involved if any of the candidates are ready in PyPI.
Correct: we're on track to releasing the fix in 1.78.1. Unless something unexpected happens, it'll be out this week or early next week.
CC @asheshvidyut
Activity
Limit ydb to < 3.19.0
Limit ydb to < 3.19.0 (#47105)
Limit ydb to < 3.19.0 (apache#47105)
liferoad commentedon Feb 28, 2025
https://github.com/apache/beam/blob/master/sdks/python/setup.py#L358 and grpc/grpc#37710
liferoad commentedon Feb 28, 2025
cc @tvalentyn
tvalentyn commentedon Mar 1, 2025
grpc/grpc#37710 was never rootcaused; we could try to lift the restriction, and see if the issue is present in recent grpc versions.
potiuk commentedon Mar 1, 2025
Sounds reasonable indeed. Thanks for looking at it!
tvalentyn commentedon Mar 24, 2025
There are at least two issue with new versions of GRPC:
ping timeouterrors: SDK->Runner communication channel sometimes breaks down when the pipeline is running large volumes of data over the network (I have a repro that involves writing ~50GB of dummy data to GCS).We are investigating this issues in close contact with GRPC team but would not recommend updating GRPC until this is rootcaused.
33 remaining items
tvalentyn commentedon Feb 16, 2026
@damccorm @Amar3tto let's try to have this fixed in 2.72.0, given that we are only waiting for the cherrypick release for grpc/grpc#37710 . Let's work with @sergiitk 's team if we must delay the release for a couple of days until grpc's release is available and tested.
Thanks!
FYI @potiuk @sergiitk .
damccorm commentedon Feb 17, 2026
@sergiitk do you have a timeline on when this will be done by?
tvalentyn commentedon Feb 17, 2026
I spoke with @sergiitk offline, the team expects to start the release in the next 24 hrs, and it might take 1-2 days to finish. @sergiitk feel free to correct. thanks.
potiuk commentedon Feb 17, 2026
Thanks for the info! Really looking forward to it. And also from the testing perspective - airflow has quite a comprehensive overall testing harness on various levels, and we will be absolutely happy to kick it in and run the full suite of our tests - from complex cross-dependency resolution of things to actual operator tests that can use release candidates when ready - we can make PRs with updated dependencies to rc candidates and run full test suite very easily, so I am super happy to get involved if any of the candidates are ready in PyPI.
tvalentyn commentedon Feb 17, 2026
We will unpin the version in anticipation of grpc==1.78.1 to be available and fixing the issue: #37633
That should allow Beam users to install newer versions of grpc once they become available.
sergiitk commentedon Feb 19, 2026
Correct: we're on track to releasing the fix in 1.78.1. Unless something unexpected happens, it'll be out this week or early next week.
CC @asheshvidyut
potiuk commentedon Feb 19, 2026
Cool :)
Limit ydb to < 3.19.0 (#47105)
potiuk commentedon Feb 28, 2026
Not sure if you realized already but grpcio 1.78.1 has been yanked due to not working with serverless environments. grpc/grpc#41725
Limit ydb to < 3.19.0 (#47105)
Limit ydb to < 3.19.0 (#47105)
Limit ydb to < 3.19.0 (#47105)