Repository navigation
feast>=0.25.1 breaks compatibility with protobuf<3.20 #3287
Description
Expected Behavior
setup.py has protobuf<5,>3, so I would expect version of protobuf in that range to work. More concretely, tensorflow tensorflow/tensorflow@60d5bfb requires protobuf >= 3.9.2, < 3.20' and it is reasonable to want to use tensorflow and feast together.
I first noticed this with the 0.26.0 release, but 0.25.1 is when I can first reproduce the problem.
Current Behavior
feast 0.25.0
$ pip install feast==0.25.0
...
$ feast version
Feast SDK Version: "feast 0.25.0"
$ pip install protobuf~=3.19.5
Collecting protobuf~=3.19.5
Using cached protobuf-3.19.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)
Installing collected packages: protobuf
Attempting uninstall: protobuf
Found existing installation: protobuf 3.20.3
Uninstalling protobuf-3.20.3:
Successfully uninstalled protobuf-3.20.3
Successfully installed protobuf-3.19.6
$ feast version
Feast SDK Version: "feast 0.25.0"
feast 0.25.1
$ pip install feast==0.25.1
$ feast version
Feast SDK Version: "feast 0.25.1"
$ pip install protobuf~=3.19.5
Collecting protobuf~=3.19.5
Using cached protobuf-3.19.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)
Installing collected packages: protobuf
Attempting uninstall: protobuf
Found existing installation: protobuf 3.20.3
Uninstalling protobuf-3.20.3:
Successfully uninstalled protobuf-3.20.3
Successfully installed protobuf-3.19.6
$ feast version
Traceback (most recent call last):
File "/tmp/proto/f251/env/bin/feast", line 5, in <module>
from feast.cli import cli
File "/tmp/proto/f251/env/lib/python3.10/site-packages/feast/__init__.py", line 7, in <module>
from feast.infra.offline_stores.bigquery_source import BigQuerySource
File "/tmp/proto/f251/env/lib/python3.10/site-packages/feast/infra/offline_stores/bigquery_source.py", line 5, in <module>
from feast import type_map
File "/tmp/proto/f251/env/lib/python3.10/site-packages/feast/type_map.py", line 37, in <module>
from feast.protos.feast.types.Value_pb2 import (
File "/tmp/proto/f251/env/lib/python3.10/site-packages/feast/protos/feast/types/Value_pb2.py", line 5, in <module>
from google.protobuf.internal import builder as _builder
ImportError: cannot import name 'builder' from 'google.protobuf.internal' (/tmp/proto/f251/env/lib/python3.10/site-packages/google/protobuf/internal/__init__.py)
Specifications
- Version:
Feast SDK Version: "feast 0.25.1" - Platform:
Linux 5.4.x - Subsystem:
x86_64
Some history in #3103
Reacted by John Foley
Activity
psheorangithub commentedon Oct 20, 2022
I'm also facing the same issue.Are there any plans to fix it?
cburroughs commentedon Jan 24, 2023
I looked at this a bit more and I think it is a variant of protocolbuffers/protobuf#9778
Code generated with version X is only guaranteed to work with clients X>=1. So the minimal version of protobuf supported by feast becomes whatever version was used in in CI to generate, not the version in
setup.py. I think the solution would look something like "compile-protos-pythonuses a separate environment with a fixed version of protobuf". I'm not sure if Makefile changes or github action configs would be the more palatable approach to doing that.