Skip to content

Paths to Param Files in Command Lines Refer to Host Execroot Paths (instead of sandbox execroot paths) With --incompatible_sandbox_hermetic_tmp=true #20515

New issue
New issue
Closed
Closed
Paths to Param Files in Command Lines Refer to Host Execroot Paths (instead of sandbox execroot paths) With --incompatible_sandbox_hermetic_tmp=true#20515
Assignees
lberki
Labels
team-Local-ExecIssues and PRs for the Execution (Local) teamtype: bug
@rrbutani

Description

@rrbutani
rrbutani
opened on Dec 12, 2023

Issue

Building Bazel at HEAD with Bazel 7.0.0 (with --incompatible_sandbox_hermetic_tmp=true, as is the default in Bazel 7) and an output base on /tmp yields the following error about a .params file not being found:

❯ bazel-7.0.0-linux-x86_64 --output_user_root=/tmp/rrbutani-bazel build //src:bazel -c opt
ERROR: /src/dev/bazel/src/main/java/com/google/devtools/build/lib/analysis/BUILD:153:13: Compiling Java headers src/main/java/com/google/devtools/build/lib/analysis/libanalysis_cluster-hjar.jar (109 source files) failed: (Exit 1): linux-sandbox failed: error executing Turbine command
  (cd /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/execroot/_main && \
  exec env - \
    LC_CTYPE=en_US.UTF-8 \
    PATH=/usr/local/bin:/usr/bin:/bin \
    TMPDIR=/tmp \
  /tmp/bazel-rrbutani/install/89a68939cbf63eb54205fdf943a58b15/linux-sandbox -W /tmp/bazel-working-directory/_main -t 15 -w /tmp/bazel-execroot/_main -w /tmp -w /dev/shm -M /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/execroot -m /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-execroot -M /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/execroot -m /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-working-directory -M /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/external/rules_java~7.3.1~toolchains~remote_java_tools_linux -m /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-source-roots/0 -M /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/external/rules_java~7.3.1~toolchains~remotejdk21_linux -m /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-source-roots/1 -M /src/dev/bazel -m /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-source-roots/2 -M /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp -m /tmp -S /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/stats.out -D /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/debug.out -- external/rules_java~7.3.1~toolchains~remote_java_tools_linux/java_tools/turbine_direct_graal '-Dturbine.ctSymPath=external/rules_java~7.3.1~toolchains~remotejdk21_linux/lib/ct.sym' @bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libanalysis_cluster-hjar.jar-0.params)
java.lang.AssertionError: params file does not exist: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libanalysis_cluster-hjar.jar-0.params
        at com.google.turbine.options.TurbineOptionsParser.expandParamsFiles(TurbineOptionsParser.java:182)
        at com.google.turbine.options.TurbineOptionsParser.parse(TurbineOptionsParser.java:49)
        at com.google.turbine.options.TurbineOptionsParser.parse(TurbineOptionsParser.java:38)
        at com.google.turbine.main.Main.compile(Main.java:133)
        at com.google.turbine.main.Main.main(Main.java:89)
Target //src:bazel failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1.225s, Critical Path: 0.15s
INFO: 3 processes: 3 internal.
ERROR: Build did NOT complete successfully

If --incompatible_sandbox_hermetic_tmp=false is passed, the above command succeeds.

Details

Running with --sandbox_debug produces the following:

Click to expand 
DEBUG: Sandbox debug output for Turbine //src/main/java/com/google/devtools/build/lib/analysis:analysis_cluster: 1702418703.999694695: src/main/tools/linux-sandbox.cc:156: calling pipe(2)...
1702418703.999718491: src/main/tools/linux-sandbox.cc:165: Netns is 0
1702418703.999720291: src/main/tools/linux-sandbox.cc:176: calling clone(2)...
1702418704.000568193: src/main/tools/linux-sandbox.cc:185: linux-sandbox-pid1 has PID 30502
1702418704.000590115: src/main/tools/linux-sandbox-pid1.cc:700: Pid1Main started
1702418704.000652861: src/main/tools/linux-sandbox.cc:202: done manipulating pipes
1702418704.000775265: src/main/tools/linux-sandbox-pid1.cc:302: bind mount: /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/execroot -> /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-execroot
1702418704.000791794: src/main/tools/linux-sandbox-pid1.cc:302: bind mount: /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/execroot -> /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-working-directory
1702418704.000799418: src/main/tools/linux-sandbox-pid1.cc:302: bind mount: /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/external/rules_java~7.3.1~toolchains~remote_java_tools_linux -> /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-source-roots/0
1702418704.000809258: src/main/tools/linux-sandbox-pid1.cc:302: bind mount: /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/external/rules_java~7.3.1~toolchains~remotejdk21_linux -> /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-source-roots/1
1702418704.000815958: src/main/tools/linux-sandbox-pid1.cc:302: bind mount: /src/dev/bazel -> /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-source-roots/2
1702418704.000834430: src/main/tools/linux-sandbox-pid1.cc:302: bind mount: /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp -> /tmp
1702418704.000848407: src/main/tools/linux-sandbox-pid1.cc:311: writable: /tmp/bazel-execroot/_main
1702418704.000854112: src/main/tools/linux-sandbox-pid1.cc:311: writable: /tmp
1702418704.000869304: src/main/tools/linux-sandbox-pid1.cc:311: writable: /dev/shm
1702418704.000876579: src/main/tools/linux-sandbox-pid1.cc:327: working dir: /tmp/bazel-working-directory/_main
1702418704.000949533: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /
1702418704.000959313: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev
1702418704.000963615: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /dev/shm
1702418704.000967575: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev/pts
1702418704.000971798: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev/hugepages
1702418704.000975643: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev/mqueue
1702418704.000979660: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /proc
1702418704.000993590: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys
1702418704.001141631: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run
1702418704.001206959: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /var
1702418704.001215631: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /tmp
<unrelated paths snipped>
1702418704.002958323: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-execroot
1702418704.002966211: src/main/tools/linux-sandbox-pid1.cc:427: remount(nullptr, /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-execroot, nullptr, 2101281, nullptr) failure (No such file or directory) ignored
1702418704.002970402: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-working-directory
1702418704.002972290: src/main/tools/linux-sandbox-pid1.cc:427: remount(nullptr, /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-working-directory, nullptr, 2101281, nullptr) failure (No such file or directory) ignored
1702418704.002974770: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-source-roots/0
1702418704.002976413: src/main/tools/linux-sandbox-pid1.cc:427: remount(nullptr, /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-source-roots/0, nullptr, 2101281, nullptr) failure (No such file or directory) ignored
1702418704.002988558: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-source-roots/1
1702418704.002990230: src/main/tools/linux-sandbox-pid1.cc:427: remount(nullptr, /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-source-roots/1, nullptr, 2101281, nullptr) failure (No such file or directory) ignored
1702418704.002997175: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-source-roots/2
1702418704.002998951: src/main/tools/linux-sandbox-pid1.cc:427: remount(nullptr, /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-source-roots/2, nullptr, 2101281, nullptr) failure (No such file or directory) ignored
1702418704.003001364: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /tmp
1702418704.003004727: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-execroot
1702418704.003008764: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-working-directory
1702418704.003012225: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-source-roots/0
1702418704.003015831: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-source-roots/1
1702418704.003029328: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-source-roots/2
1702418704.003033205: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /tmp/bazel-execroot/_main
1702418704.003036504: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /tmp
1702418704.003039277: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-execroot
1702418704.003042065: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /tmp/bazel-execroot/_main
1702418704.003044940: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-working-directory
1702418704.003047774: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-source-roots/0
1702418704.003050505: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-source-roots/1
1702418704.003057570: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-source-roots/2
1702418704.003060582: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /dev/shm
1702418704.003064295: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /tmp/bazel-working-directory/_main
1702418704.003087515: src/main/tools/linux-sandbox-pid1.cc:496: calling fork...
1702418704.003209294: src/main/tools/linux-sandbox-pid1.cc:533: child started with PID 2
1702418704.005588552: src/main/tools/linux-sandbox-pid1.cc:550: wait returned pid=2, status=0x100
1702418704.005591954: src/main/tools/linux-sandbox-pid1.cc:568: child exited normally with code 1
1702418704.034940719: src/main/tools/linux-sandbox.cc:243: child exited normally with code 1

Inspecting the mounts and their ordering above confirms that contents in Bazel source roots and execroot should be visible: these mounts are bind mounted into the hermetic tmp location (not /tmp directly) before /tmp is pivoted to the hermetic tmp location (the final bind mount: in the above).

By altering the linux-sandbox invocation above we can confirm that the bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libanalysis_cluster-hjar.jar-0.params file is a dangling symlink, within the sandbox:

/tmp/bazel-rrbutani/install/89a68939cbf63eb54205fdf943a58b15/linux-sandbox \
    -W /tmp/bazel-working-directory/_main \
    -t 15 \
    -w /tmp/bazel-execroot/_main \
    -w /tmp \
    -w /dev/shm \
    -M /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/execroot \
        -m /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-execroot \
    -M /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/execroot \
        -m /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-working-directory \
    -M /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/external/rules_java~7.3.1~toolchains~remote_java_tools_linux \
        -m /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-source-roots/0 \
    -M /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/external/rules_java~7.3.1~toolchains~remotejdk21_linux \
        -m /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-source-roots/1 \
    -M /src/dev/bazel \
        -m /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp/bazel-source-roots/2 \
    -M /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/_hermetic_tmp \
        -m /tmp \
    -S /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/stats.out \
    -D /dev/null \
    -- ls --color -l bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libanalysis_cluster-hjar.jar-0.params
lrwxrwxrwx 1 rrbutani users 179 Dec 12 14:05 bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libanalysis_cluster-hjar.jar-0.params -> /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/execroot/_main/bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libanalysis_cluster-hjar.jar-0.params

Inspecting bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libanalysis_cluster-hjar.jar-0.params within this action's execroot (/tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/execroot/) confirms that the file is a symlink that refers to the host's execroot path rather than that of the sandbox (i.e. /tmp/bazel-execroot/):

❯ ls /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/execroot/_main/bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libanalysis_cluster-hjar.jar-0.params -l
lrwxrwxrwx 1 rrbutani users 179 Dec 12 14:05 /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/execroot/_main/bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libanalysis_cluster-hjar.jar-0.params -> /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/execroot/_main/bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libanalysis_cluster-hjar.jar-0.params

❯ ls /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3031/execroot/_main/bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/ -l | head
total 16
drwxr-x--- 2 rrbutani users 4096 Dec 12 14:05 libactions
lrwxrwxrwx 1 rrbutani users  128 Dec 12 14:05 libactions_provider-hjar.jar -> /tmp/bazel-execroot/_main/bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libactions_provider-hjar.jar
lrwxrwxrwx 1 rrbutani users  179 Dec 12 14:05 libanalysis_cluster-hjar.jar-0.params -> /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/execroot/_main/bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libanalysis_cluster-hjar.jar-0.params
lrwxrwxrwx 1 rrbutani users  141 Dec 12 14:05 libaspect_aware_attribute_mapper-hjar.jar -> /tmp/bazel-execroot/_main/bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libaspect_aware_attribute_mapper-hjar.jar
lrwxrwxrwx 1 rrbutani users  129 Dec 12 14:05 libaspect_collection-hjar.jar -> /tmp/bazel-execroot/_main/bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libaspect_collection-hjar.jar
lrwxrwxrwx 1 rrbutani users  130 Dec 12 14:05 libblaze_version_info-hjar.jar -> /tmp/bazel-execroot/_main/bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libblaze_version_info-hjar.jar
lrwxrwxrwx 1 rrbutani users  134 Dec 12 14:05 libbuild_setting_provider-hjar.jar -> /tmp/bazel-execroot/_main/bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libbuild_setting_provider-hjar.jar
drwxr-x--- 3 rrbutani users 4096 Dec 12 14:05 libconfig
lrwxrwxrwx 1 rrbutani users  129 Dec 12 14:05 libconfigured_target-hjar.jar -> /tmp/bazel-execroot/_main/bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libconfigured_target-hjar.jar

(note how the other files in the above are symlinks to /tmp/bazel-execroot/)

Cause

Poking around a bit, I think this is the relevant part of rules_java:

bazel/src/main/java/com/google/devtools/build/lib/rules/java/JavaHeaderCompileAction.java

Lines 486 to 489 in 379ee5f

/* commandLines= */ CommandLines.builder()
.addCommandLine(executableLine)
.addCommandLine(commandLine.build(), PARAM_FILE_INFO)
.build(),

Notably rules_cc generated param files for link actions do not seem to have this issue (note the /tmp/bazel-execroot in the destinations):

❯ fd . /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/ | grep params | xargs ls -l --color
lrwxrwxrwx 1 rrbutani users 179 Dec 12 14:25 /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3006/execroot/_main/bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libanalysis_cluster-hjar.jar-0.params -> /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/execroot/_main/bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/analysis/libanalysis_cluster-hjar.jar-0.params
lrwxrwxrwx 1 rrbutani users  80 Dec 12 14:27 /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3098/execroot/_main/bazel-out/k8-dbg/bin/src/main/tools/daemonize-2.params -> /tmp/bazel-execroot/_main/bazel-out/k8-dbg/bin/src/main/tools/daemonize-2.params
lrwxrwxrwx 1 rrbutani users  83 Dec 12 14:27 /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3114/execroot/_main/bazel-out/k8-dbg/bin/src/main/tools/liblogging.a-2.params -> /tmp/bazel-execroot/_main/bazel-out/k8-dbg/bin/src/main/tools/liblogging.a-2.params
lrwxrwxrwx 1 rrbutani users 111 Dec 12 14:27 /tmp/bazel-rrbutani/705c9771f27ae01ebf0993be3b99a716/sandbox/linux-sandbox/3149/execroot/_main/bazel-out/k8-dbg/bin/src/main/java/net/starlark/java/eval/libcpu_profiler.so-2.params -> /tmp/bazel-execroot/_main/bazel-out/k8-dbg/bin/src/main/java/net/starlark/java/eval/libcpu_profiler.so-2.params

I think that's because rules_cc generates its param files (for link actions) on its own using separate actions + files that are added as dependencies and doesn't use the CommandLines.Builder machinery:

bazel/src/main/java/com/google/devtools/build/lib/rules/cpp/CppLinkActionBuilder.java

Lines 950 to 971 in 0aacc74

if (linkCommandLine.getParamFile() != null) {
inputsBuilder.add(linkCommandLine.getParamFile());
// Pass along tree artifacts, so they can be properly expanded.
NestedSet<Artifact> paramFileActionInputs =
NestedSetBuilder.wrap(
Order.STABLE_ORDER,
Iterables.filter(expandedLinkerArtifacts.toList(), Artifact::isTreeArtifact));
ParameterFile.ParameterFileType quoting =
featureConfiguration.isEnabled(CppRuleClasses.GCC_QUOTING_FOR_PARAM_FILES)
? ParameterFile.ParameterFileType.GCC_QUOTED
: ParameterFile.ParameterFileType.UNQUOTED;
Action parameterFileWriteAction =
new ParameterFileWriteAction(
getOwner(),
paramFileActionInputs,
paramFile,
linkCommandLine.paramCmdLine(),
quoting);
actionConstructionContext.registerAction(parameterFileWriteAction);
}

I am not familiar with this part of the Bazel codebase but it seems likely the problematic bit (placing host-execroot paths for the params file onto the command line) lives here:

bazel/src/main/java/com/google/devtools/build/lib/actions/CommandLines.java

Lines 121 to 154 in 29f1db2

if (useParamFile) {
PathFragment paramFileExecPath =
ParameterFile.derivePath(paramFileBasePath, Integer.toString(paramFileNameSuffix));
++paramFileNameSuffix;
String paramArg =
SingleStringArgFormatter.format(
paramFileInfo.getFlagFormatString(),
pathMapper.map(paramFileExecPath).getPathString());
arguments.addElement(paramArg);
cmdLineLength += paramArg.length() + 1;
if (paramFileInfo.flagsOnly()) {
// Move just the flags into the file, and keep the positional parameters on the command
// line.
paramFiles.add(
new ParamFileActionInput(
paramFileExecPath,
ParameterFile.flagsOnly(args),
paramFileInfo.getFileType(),
paramFileInfo.getCharset()));
for (String positionalArg : ParameterFile.nonFlags(args)) {
arguments.addElement(positionalArg);
cmdLineLength += positionalArg.length() + 1;
}
} else {
paramFiles.add(
new ParamFileActionInput(
paramFileExecPath,
args,
paramFileInfo.getFileType(),
paramFileInfo.getCharset()));
}
}

And here:

bazel/src/main/java/com/google/devtools/build/lib/analysis/actions/SpawnAction.java

Lines 348 to 360 in 41ca39a

ExpandedCommandLines expandedCommandLines =
commandLines.expand(
artifactExpander, getPrimaryOutput().getExecPath(), pathMapper, getCommandLineLimits());
return new ActionSpawn(
ImmutableList.copyOf(expandedCommandLines.arguments()),
this,
env,
envResolved,
getInputs(),
expandedCommandLines.getParamFiles(),
filesetMappings,
reportOutputs,
pathMapper);

(note the use of getPrimaryOutput().getExecPath() for paramFileBasePath)

Metadata

Metadata

Assignees

Labels

team-Local-ExecIssues and PRs for the Execution (Local) teamtype: bug

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions