Manifest-based runfiles lookup is non-hermetic #14500
Description
Description of the problem / feature request:
The Bazel runfiles libraries behave non-hermetically with --enable_runfiles=false.
Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
- Clone https://github.com/fmeum/bazel-non-hermetic-runfiles
- Run
bazel run //parent:Parentand observe that the runfiles lookup succeeds. - Comment out the
datadependency of//child. - Run
bazel build //child. - Revert the change made in step 3.
- Do not run
bazel build //child. - Run
bazel run //parent:Parentand observe that the runfiles lookup fails.
What operating system are you running Bazel on?
Linux, but applies to all OSes. This is particularly important on Windows where --enable_runfiles defaults to false.
What's the output of bazel info release?
5.0.0rc3
Have you found anything relevant by searching the web?
The change that introduced the regression is bfdfa6e. The same behavior can't be reproduced with Bazel 4.2.2, since the runfiles env variables are checked first there before the runfiles manifest is looked up next to the binary.
The root cause of the issue is that the first manifest-based lookup returns a path in the Bazel cache. But the contents of the cache are not hermetic and may contain outdated runfiles manifests for non-top-level binaries. If any of these binaries is invoked and first searches for the manifest next to itself instead of using the manifest set in RUNFILES_MANIFEST_FILE, it will behave non-hermetically.
The same issue can be reproduced with --enable_runfiles=true if for some reason the parent process resolves the runfiles tree symlink to the child process before launching it.
The process described in the original design doc is also affected. I do not know whether it has been updated after the breaking commit or has always been buggy in this way.
Any other information, logs, or outputs that you want to share?
The output of the reproducer with Bazel 4.2.2:
Runfiles variables in parent:
RUNFILES_MANIFEST_FILE=/home/fhenneke/.cache/bazel/_bazel_fhenneke/58c66360f3011c1a109b2b3996e84c32/execroot/__main__/bazel-out/k8-fastbuild/bin/parent/Parent.runfiles/MANIFEST
RUNFILES_DIR=/home/fhenneke/.cache/bazel/_bazel_fhenneke/58c66360f3011c1a109b2b3996e84c32/execroot/__main__/bazel-out/k8-fastbuild/bin/parent/Parent.runfiles
JAVA_RUNFILES=/home/fhenneke/.cache/bazel/_bazel_fhenneke/58c66360f3011c1a109b2b3996e84c32/execroot/__main__/bazel-out/k8-fastbuild/bin/parent/Parent.runfiles
RUNFILES_MANIFEST_ONLY=1
Starting child: /home/fhenneke/.cache/bazel/_bazel_fhenneke/58c66360f3011c1a109b2b3996e84c32/execroot/__main__/bazel-out/k8-fastbuild/bin/child/child
Child started with argv[0]=/home/fhenneke/.cache/bazel/_bazel_fhenneke/58c66360f3011c1a109b2b3996e84c32/execroot/__main__/bazel-out/k8-fastbuild/bin/child/child
Runfiles variables in child:
RUNFILES_MANIFEST_FILE=/home/fhenneke/.cache/bazel/_bazel_fhenneke/58c66360f3011c1a109b2b3996e84c32/execroot/__main__/bazel-out/k8-fastbuild/bin/parent/Parent.runfiles/MANIFEST
RUNFILES_DIR=/home/fhenneke/.cache/bazel/_bazel_fhenneke/58c66360f3011c1a109b2b3996e84c32/execroot/__main__/bazel-out/k8-fastbuild/bin/parent/Parent.runfiles
JAVA_RUNFILES=/home/fhenneke/.cache/bazel/_bazel_fhenneke/58c66360f3011c1a109b2b3996e84c32/execroot/__main__/bazel-out/k8-fastbuild/bin/parent/Parent.runfiles
Found runfile at: /home/fhenneke/git/bazel-non-hermetic-runfiles/data/foo.txt
The output of the last step of the reproducer with Bazel 5.0.0rc3:
Runfiles variables in parent:
RUNFILES_MANIFEST_FILE=/home/fhenneke/.cache/bazel/_bazel_fhenneke/58c66360f3011c1a109b2b3996e84c32/execroot/__main__/bazel-out/k8-fastbuild/bin/parent/Parent.runfiles/MANIFEST
RUNFILES_DIR=/home/fhenneke/.cache/bazel/_bazel_fhenneke/58c66360f3011c1a109b2b3996e84c32/execroot/__main__/bazel-out/k8-fastbuild/bin/parent/Parent.runfiles
JAVA_RUNFILES=/home/fhenneke/.cache/bazel/_bazel_fhenneke/58c66360f3011c1a109b2b3996e84c32/execroot/__main__/bazel-out/k8-fastbuild/bin/parent/Parent.runfiles
RUNFILES_MANIFEST_ONLY=1
Starting child: /home/fhenneke/.cache/bazel/_bazel_fhenneke/58c66360f3011c1a109b2b3996e84c32/execroot/__main__/bazel-out/k8-fastbuild/bin/child/child
Child started with argv[0]=/home/fhenneke/.cache/bazel/_bazel_fhenneke/58c66360f3011c1a109b2b3996e84c32/execroot/__main__/bazel-out/k8-fastbuild/bin/child/child
Runfiles variables in child:
RUNFILES_MANIFEST_FILE=/home/fhenneke/.cache/bazel/_bazel_fhenneke/58c66360f3011c1a109b2b3996e84c32/execroot/__main__/bazel-out/k8-fastbuild/bin/child/child.runfiles/MANIFEST
RUNFILES_DIR=/home/fhenneke/.cache/bazel/_bazel_fhenneke/58c66360f3011c1a109b2b3996e84c32/execroot/__main__/bazel-out/k8-fastbuild/bin/child/child.runfiles
JAVA_RUNFILES=/home/fhenneke/.cache/bazel/_bazel_fhenneke/58c66360f3011c1a109b2b3996e84c32/execroot/__main__/bazel-out/k8-fastbuild/bin/child/child.runfiles
Failed to find runfile at: /home/fhenneke/.cache/bazel/_bazel_fhenneke/58c66360f3011c1a109b2b3996e84c32/execroot/__main__/bazel-out/k8-fastbuild/bin/child/child.runfiles/__main__/data/foo.txt