Skip to content

[Bug]: gprcio limitation to < 1.66 in Python is problematic #34081

Description

@potiuk

What happened?

Some of the dependencies of Airflow (ydb) started to expect grpcio > 1.68.1 and apache-beam has < 1.66 - this is causing us to limit the other packages from installing - which we do not really like to do. It would be great to lift the upper-bound limitation if possible.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner

Activity

  1. added a commit that references this issue on Feb 28, 2025
  2. liferoad commented on Feb 28, 2025

    @liferoad
    Contributor
  3. tvalentyn commented on Mar 1, 2025

    @tvalentyn
    Contributor

    grpc/grpc#37710 was never rootcaused; we could try to lift the restriction, and see if the issue is present in recent grpc versions.

  4. potiuk commented on Mar 1, 2025

    @potiuk
    MemberAuthor

    grpc/grpc#37710 was never rootcaused; we could try to lift the restriction, and see if the issue is present in recent grpc versions.

    Sounds reasonable indeed. Thanks for looking at it!

  5. added this to the 2.65.0 Release milestone on Mar 1, 2025
  6. self-assigned this
    on Mar 4, 2025
  7. tvalentyn commented on Mar 24, 2025

    @tvalentyn
    Contributor

    There are at least two issue with new versions of GRPC:

    1. Our Go tests were failing on grpc 1.66.0 and newer
    2. Using grpcio==1.68.0, and some newer versions (haven't checked the latest version), causes pipeline failures due to ping timeout errors: SDK->Runner communication channel sometimes breaks down when the pipeline is running large volumes of data over the network (I have a repro that involves writing ~50GB of dummy data to GCS).

    We are investigating this issues in close contact with GRPC team but would not recommend updating GRPC until this is rootcaused.

  8. 33 remaining items

  9. tvalentyn commented on Feb 16, 2026

    @tvalentyn
    Contributor

    @damccorm @Amar3tto let's try to have this fixed in 2.72.0, given that we are only waiting for the cherrypick release for grpc/grpc#37710 . Let's work with @sergiitk 's team if we must delay the release for a couple of days until grpc's release is available and tested.

    Thanks!

    FYI @potiuk @sergiitk .

  10. damccorm commented on Feb 17, 2026

    @damccorm
    Contributor

    @sergiitk do you have a timeline on when this will be done by?

  11. tvalentyn commented on Feb 17, 2026

    @tvalentyn
    Contributor

    I spoke with @sergiitk offline, the team expects to start the release in the next 24 hrs, and it might take 1-2 days to finish. @sergiitk feel free to correct. thanks.

  12. potiuk commented on Feb 17, 2026

    @potiuk
    MemberAuthor

    Thanks for the info! Really looking forward to it. And also from the testing perspective - airflow has quite a comprehensive overall testing harness on various levels, and we will be absolutely happy to kick it in and run the full suite of our tests - from complex cross-dependency resolution of things to actual operator tests that can use release candidates when ready - we can make PRs with updated dependencies to rc candidates and run full test suite very easily, so I am super happy to get involved if any of the candidates are ready in PyPI.

  13. tvalentyn commented on Feb 17, 2026

    @tvalentyn
    Contributor

    We will unpin the version in anticipation of grpc==1.78.1 to be available and fixing the issue: #37633

    That should allow Beam users to install newer versions of grpc once they become available.

  14. sergiitk commented on Feb 19, 2026

    @sergiitk
    Contributor

    Correct: we're on track to releasing the fix in 1.78.1. Unless something unexpected happens, it'll be out this week or early next week.
    CC @asheshvidyut

  15. potiuk commented on Feb 19, 2026

    @potiuk
    MemberAuthor

    Cool :)

  16. added a commit that references this issue on Feb 27, 2026
  17. potiuk commented on Feb 28, 2026

    @potiuk
    MemberAuthor

    Not sure if you realized already but grpcio 1.78.1 has been yanked due to not working with serverless environments. grpc/grpc#41725

  18. added a commit that references this issue on Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions