Hello @jmillikin, modules you maintain (glib) have been updated in this PR.
Please review the changes. You can view a diff against the previous version in the "Generate module diff" check.

note, I am not a toolchain expert, so it would be good if someone looked at it carefully.

The tools to execute are defined with @rules_python//python:defs.bzl%py_binary, so if the Python toolchain is configured correctly then they should be executable as-is. If something within Bazel is going wrong on your system, then working around that with a genrule (which just hardcodes /bin/bash as the executable) doesn't seem like a great approach.

Using a genrule would be appropriate if for some reason the Python toolchain configured by Bazel is inappropriate and the system version needs to be used, but that doesn't seem to be the case here.

If a simple py_binary plus run_binary can't be executed on your system, then I recommend filing a bug against rules_python (https://github.com/bazel-contrib/rules_python). A small reproducible example would be easier for them to debug than the glib module.

Filed bazel-contrib/rules_python#3575

Using a genrule would be appropriate if for some reason the Python toolchain configured by Bazel is inappropriate and the system version needs to be used, but that doesn't seem to be the case here.

In the genrule(), I am not using the system version: with the $(PYTHON3) I am invoking the python interpreter provided by the toolchains = ["@rules_python//python:current_py_toolchain"], which sets that variable.

The run_binary() instead looks like it just attempts to use the system version (by evaluating the #!) - this is why it is failing on a system where there is no python installed or bazel prevents /usr/bin/env to read the PATH environment variable.

Possible idea -- if rules_python uses a precompiled Python toolchain, and NixOS uses a non-standard location of libc.so or other dynamic libraries, then the Linux kernel will report No such file or directory to mean that the executable couldn't be fully loaded. So possibly what's needed is some LD_LIBRARY_PATH plumbing.

run_binary is a thin wrapper around ctx.actions.run, which delegates to the OS's logic for spawning subprocesses. The logic of which Python interpreter gets executed (/usr/bin/python3 or toolchain-provided) needs to exist within the file generated by py_binary.

I tried following the logic within rules_python to figure out how it goes from the toolchain to the generated shim, but that path has a lot more complexity in it than I was expecting. I think the relevant code is somewhere in the call tree of @rules_python//python/private:py_executable.bzl%_create_executable. It should end up emitting a small shim that has the path of the Python interpreter in it.

You could try cating the intermediate files generated by Bazel to see if it's mis-detecting some aspect of your system.

For comparison/reference, this is the code I use in my own projects for something similar (just JS instead of Python). It just calls exec with the path to the nodejs binary. I would expect rules_python to have similar code somewhere.

def _nodejs_binary(ctx):
    nodejs = nodejs_toolchain(ctx)
    out = ctx.actions.declare_file(ctx.attr.name)
    ctx.actions.write(
        output = out,
        content = """#!/bin/sh
exec "{node_js}" -- "{main_js}" "$@"
""".format(
            node_js = nodejs.node_tool.executable.path,
            main_js = ctx.file.src.path,
        ),
    )

From the on the rules_python bug I filed it looks like that the python toolchain uses the system python too bootstrap, which will not work if there is no Python. They also seem to have an alternative like the /bin/sh launcher you showed, but it is using bash, which is of course also a non-standard shell. So rules_python is not ready yet for run_binary() a py_binary().

The genrule using the output of py_binary() is also not recommended as that could not necessarily be a raw python script anymore, depending on the prefix are added.
So the best solution is to use the genrule(), but use the raw python script, and not use the output of py_binary()

If ok by you, I'll change this PR using that technique.

Given the complexity of the GLib compilation process, I'd prefer to stick to using standard Bazel machinery (py_binary, run_binary, etc) instead of ad-hoc genrules. Requiring /usr/bin/python to locate the Bazel-managed toolchain Python is ... not the choice I would have made, but the rules_python developers have more context on the tradeoffs involved there.

I'm sympathetic to the ideal of builds that use only the bare minimum of system dependencies (e.g. POSIX-guaranteed /bin/sh instead of /bin/bash), but as a practical matter I think it's higher priority to have a reliable build on the important platforms (mainstream Linux + Windows + macOS). The Bazel architecture has not historically prioritized hermeticity other than as a side effect of seeking determinism.

Note in particular that Bazel in general has a hard dependency on system Bash (or a Bash-compatible) shell, genrule itself expects Unix platforms to use Bash, and the --shell_executable option has a clear warning about non-Bash shells causing build failures. There are many third-party modules that assume all non-Windows platforms have Bash available.

Finally, even if you used workarounds to get this one version working in NixOS, there's no CI coverage for that platform in BCR. Any future refactoring might break it without anybody noticing.

Aside: I notice in the linked ticket that the rules_python Bash-based launcher doesn't work on your system because env couldn't be executed, which also seems fairly unusual. Does /usr/bin/env exist, and is /usr/bin in your $PATH?

If you can fix that error, then I believe you can set the py_binary launcher for all targets in your build graph by putting the following setting into your ~/.bazelrc (or /etc/bazel.bazelrc):

build --@rules_python//python/config_settings:bootstrap_impl=script

Then you won't need to individually patch module dependencies to support NixOS.

The --@rules_python//python/config_settings:bootstrap_impl=script also does not work.
The problem now is that I can't compile anything that depends on glib, so getting things working on NixOS (which is more and more common) is crucial.

Using the robust choice with a genrule() to directly invoke the script , and that works, vs. going a roundabout way of py_binary(), run_binary() with a very leaky abstraction in bazel that therefore does not work ... I'd prefer the simple and robust one.

I am working on getting nix added to the bazel CI separately.

cc @bazelbuild/bcr-maintainers for visibility + guidance regarding genrule vs py_binary + run_binary

+1 asking for guidance.
I think bazel and/or rules_python need to change before it is safe to use py_binary() + run_binary() and the only portable way is to use genrule right now unfortunately.
But maybe bcr maintainers can suggest a different trick that works on hermetic systems.

Disclaimer: I have no expertise in this area and can't give any guidance. But I see bazel-contrib/rules_python#3575 is fixed. Does that enable using py_binary now?

cc also @rickeylev in case he has anything to say about this.

I think that change will not fix this one (there is still the assumption of a Python existing on the system; this change uses the Python coming from the toolchain, so makes it more hermetic)

Yes, this is basically working around a lack of strict-posix compliance in rules_python. NixOS much more strictly enforces such compliance, and rules_python's py_binary is either a python script (which requires /usr/bin/env python3 to work), or a shell script (which requires a couple coreutils and bash). Neither of those is available by default on NixOS.

However, in this case, because it's just a single python file to execute without any dependencies, the orchestration logic of py_binary isn't strictly needed. A py_binary's main job is to deal with dependencies, edge cases, and features to run an arbitrary program. A simple <python> <script.py> invocation works fine. Using a genrule + toolchain as done here is perfectly fine.