Flip incompatible_strict_action_env#26587
Conversation
|
I guess now I need someone from the bazel team to trigger https://github.com/bazelbuild/continuous-integration/blob/337638d2fcff6d86b66035901c09d37c9199b38a/buildkite/bazelci.py#L99 so that we can see all downstream projects. |
05a780e to
fe4238b
Compare
|
I also sent a PR to add the flag to the BCR pipeline: bazelbuild/bazel-central-registry#5319 |
Thanks @fmeum. Do we actually have a baseline for the downstream job? I was looking at a few of the failures showing up and a significant number fails because of |
|
I don't see any failures that I would attribute to the flip, but it's true that downstream is in bad shape. That makes the signal worse. Ideally we would be able to run a downstream test with Bazel 8 and the flag flipped instead, which should avoid most of the preexisting failures. @fweikert Is that possible? |
|
@fweikert ping |
|
I re-opened #6648 and added But based on historical experience, this one is very hard to flip as many actions do depend on the value of |
|
Thanks. Let's see what we get tomorrow.
Depending how many fixed we would have to do, how about setting |
Yeah, this would work for leaf projects, but if a common dependency actually needs this flag, we'll still need to fix it and migrate the ecosystem first. Let's see the result, I started a manual run at https://buildkite.com/bazel/bcr-bazel-compatibility-test/builds/492 |
|
Looks like only rules_python might be affected on Windows, can you please take a quick look? If it's not serious, I'm fine with moving this forward and fix new breakages as we discover them. |
|
Created bazel-contrib/rules_python#3160 to reproduce this on a PR |
|
@meteorcloudy I tried to reproduce the windows error in bazel-contrib/rules_python#3160, but on rules_python CI this job passes. See https://buildkite.com/bazel/rules-python-python/builds/12739/steps/canvas?sid=0198f0e1-0e1a-4f05-8f7e-2769e21a7640#0198f0e1-0f47-4ee1-9940-61d1a48af7db/278-292 where the flag is set on the windows CI. Do you have any idea why the windows build of https://buildkite.com/bazel/bcr-bazel-compatibility-test/builds/492 could have failed? Are they running on different windows environments? |
|
I think the only difference is that in the rules_python pipeline, it's running again Bazel 7.6.1, however in the BCR pipeline, we are testing against 8.3.1. Can you try upgrade Bazel and see if it's reproducible? |
fe4238b to
453399e
Compare
|
bazel-contrib/rules_python#3160 is a fix for rules_python |
|
@bazel-io fork 9.0.0 |
Work towards bazel-contrib/SIG-rules-authors#42. Fixes bazelbuild#7026 Closes bazelbuild#26587. PiperOrigin-RevId: 831944371 Change-Id: I397faa463a709b2367d5a6eebc678077c4029bbb
Work towards bazel-contrib/SIG-rules-authors#42. Fixes #7026 Closes #26587. PiperOrigin-RevId: 831944371 Change-Id: I397faa463a709b2367d5a6eebc678077c4029bbb Commit e697ff7 Co-authored-by: Markus Hofbauer <[email protected]>
…103) We only need that for less than Bazel 9 due to bazelbuild/bazel#26587
Bazel 9 flipped `--incompatible_strict_action_env` to `true` by default (bazelbuild/bazel#26587), which means tests no longer inherit `PATH` from the host environment. This breaks subprocess calls that rely on `PATH` lookup: ``` ==================== Test output for //distro:packaging_test: E. ====================================================================== ERROR: testBuild (__main__.PackagingTest.testBuild) ---------------------------------------------------------------------- Traceback (most recent call last): [...] build_result = subprocess.check_output(['bazel', 'build', '--enable_workspace', ':dummy_tar']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [...] FileNotFoundError: [Errno 2] No such file or directory: 'bazel' ``` This change therefore adds `env_inherit = ["PATH"]` to that very test, which matches the behavior of earlier Bazel versions and is consistent with the presence of a "noci" tag. Additionally, Bazel 9 removed `WORKSPACE` support entirely (bazelbuild/bazel#26131), requiring `bzlmod` with `MODULE.bazel`: ``` ==================== Test output for //distro:packaging_test: [...] WARNING: --enable_bzlmod is set, but no MODULE.bazel file was found at the workspace root. Bazel will create an empty MODULE.bazel file. Please consider migrating your external dependencies from WORKSPACE to MODULE.bazel. For more details, please refer to bazelbuild/bazel#18958. [...] ERROR: error loading package '': Unable to find package for @@[unknown repo 'not_named_rules_pkg' requested from @@]//pkg:tar.bzl: The repository '@@[unknown repo 'not_named_rules_pkg' requested from @@]' could not be resolved: No repository visible as '@not_named_rules_pkg' from main repository. [...] ERROR: Build did NOT complete successfully E. ====================================================================== ERROR: testBuild (__main__.PackagingTest.testBuild) ---------------------------------------------------------------------- [...] build_result = subprocess.check_output(['bazel', 'build'] + bazel_flags + [':dummy_tar']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [...] subprocess.CalledProcessError: Command '['bazel', 'build', '--enable_workspace', ':dummy_tar']' returned non-zero exit status 1. ``` The test now detects the Bazel version at runtime and generates the appropriate setup: - Bazel 9+: `MODULE.bazel`, with `module` and `archive_override`, - earlier versions, as before: `WORKSPACE`, with `http_archive` and `--enable_workspace` flag. For good measure, the change also addresses a leftover TODO by replacing the `tar` subprocess invocation with pure Python code.
Work towards bazel-contrib/SIG-rules-authors#42.
Fixes #7026