-
Notifications
You must be signed in to change notification settings - Fork 11.1k
grpcio-tools and grpcio-health-checking namespace packages break grpcio in nonstandard installs #8058
Description
grpcio-health-checking currently cannot be installed separately, while using grpcio (and grpcio-tools) installed system-wide (or in any directory other than the one grpcio-health-checking is installed into).
grpcio-tools and grpcio-health-checking use "grpc" as a namespace package, which is intended to make the above work. That is: they pass namespace_packages=['grpc'] to setuptools.setup.
As documented, namespace packages have the following requirements:
- You must NOT include any other code and data in a namespace package’s
__init__.py- You must include the
declare_namespace()line in the__init__.pyof every project that has contents for the namespace package in question
grpcio-tools and grpcio-health-checking meet these requirements. However, grpcio itself uses "grpc" as a normal package: most of its code is in grpc/__init__.py, it does not call declare_namespace, and it does not have namespace_packages=['grpc'].
This mostly works, but in more unusual situations (involving installs of grpcio and the grpcio-* subpackages in different directories), it breaks a bit. For example:
- Install grpcio (and possibly grpcio-tools) into system site-packages (using favorite package manager).
- Create a virtualenv with
--system-site-packages. - Use this virtualenv's pip to install grpcio-health-checking.
If I do this using Python 3.5 and setuptools 26.1.1, it results in the virtualenv's "grpc" package shadowing the system one (because of .nspkg.pth files created in the virtualenv site-packages by setuptools). If setuptools were to change to start relying on PEP 420's implicit namespace packages instead of dropping a .nspkg.pth file, or if grpcio-health-checking was installed outside of site-packages (so .pth files are not executed), it would break in the other direction: the system grpc package gets picked up as a standard package because it has __init__.py, shadowing the __init__.py-less namespace package used by grpcio-health-checking.
I would expect pip to produce the same broken install if it is asked to install some package that depends on a grpcio-* subpackage rather than grpcio-* itself. There are probably more failing scenarios (but I'm out of practice with namespace packages, so I don't know what they all are).
Maybe the answer is "install all grpcio packages the same way in the same location to avoid these problems". But that would somewhat defeat the purpose of splitting the packages up in the first place. And especially if subpackages like grpcio-health-checking are pure Python, there is some value in being able to install them separately while using a system-wide grpcio, avoiding the need for a working C compiler and Python headers.
Possible mostly backwards-compatible fix:
- Move grpcio-tools and grpcio-health-checking to a proper namespace package that does not rely on code in
__init__.py. - Include
grpc/health.pyandgrpc/tools.pyin the main grpcio package. When imported, they import the subpackages from their new location and replace themselves. If the subpackages are not installed, you get anImportError. - Optionally, modify the forwarding modules to produce a deprecation warning on import after some deprecation period.
- After a suitable deprecation period, remove the forwarding modules from
grpc/__init__.py.