Hey all,

Problem Statement

It's quite confusing how a user can specify that a given rule need to use Python3 or that rule need Python2, where we have multiple attributes documented like default_python_version, srcs_version and undocumented like :py_interpreter (which can't be used but out of curiosity what is the prefix : doing), however I am still confused regarding what does what, in the end, what I think we should have is a way to tell py_binary to execute using the interpreter X, I know about py_runtime which is supposed to be the future but there are multiple py_binary rules out there with unspecify python version which lead to issues of python3 and 2 compatibilities, beside there is an assumption with the introduced with --python_top argument that all code need one interepreter instead of having local to each target.

Workaround

Currently we use the following workaround, maybe useful for others that suffer same issue:

• When building default to python2 since that's the safest bet and what most extensions requires.
• When running a target we dynamically switch to the needed interpreter with something similar to decorator pattern, as described below:

The Decorator Pattern

1. Create a python file with the following format, let's name it force_python3.py:

import sys
import os


PYTHON3_PATH = os.environ['PYTHON3_PATH']


if __name__ == '__main__':
    argv = sys.argv
    path, _ = os.path.split(argv[0])
    argv[0] = os.path.join(path, '<real main file>')
    argv = [PYTHON3_PATH] + argv

    os.execv(argv[0], argv)

(This assume that we expose a PYTHON3_PATH environment variable with --action_env=PYTHON3_PATH=<path>).

2. Write py_binary rule by pointing main attribute to the file above:

py_binary(
    name = "my_target",
    srcs = ["main.py", "force_python3.py"],
    main = "force_python3.py"
)

FYI we have a workaround of our own to specify which python to use for pip requirements.

Goals

What I would like to see are the followings:

• Add an attribute to py_binary and py_test rules to specify which python interpreter to use, it should be a label and point to a py_runtime, if not specified revert to default py_runtime (for backward compatibility too).
  • this means dropping probably --python_top argument b/c the assumption that you need one global Python version is bogus as explained above.
• py_runtime should be specified in WORKSPACE so that rules that download dependencies can use it, e.g. https://github.com/bazelbuild/rules_python#importing-pip-dependencies.
• Build on previous point, if py_runtime works in same say way go rules do, which use the new toolchains idea it will be awesome.
  • but keep in mind that's you may want to package the py_binary result somehow e.g. par, pex, docker image and shipping the interpreter with it will be very convenient, or at least not have a hardcoded python path (which is what you get if you point the py_runtime to an host binary path).