Skip to content

Flip incompatible_strict_action_env#26587

Closed
hofbi wants to merge 2 commits intobazelbuild:masterfrom
hofbi:flip-incompatible_strict_action_env
Closed

Flip incompatible_strict_action_env#26587
hofbi wants to merge 2 commits intobazelbuild:masterfrom
hofbi:flip-incompatible_strict_action_env

Conversation

@hofbi
Copy link
Contributor

@hofbi hofbi commented Jul 20, 2025

@hofbi
Copy link
Contributor Author

hofbi commented Jul 20, 2025

@hofbi hofbi force-pushed the flip-incompatible_strict_action_env branch 2 times, most recently from 05a780e to fe4238b Compare July 21, 2025 20:48
@fmeum
Copy link
Collaborator

fmeum commented Jul 23, 2025

@fmeum
Copy link
Collaborator

fmeum commented Jul 23, 2025

I also sent a PR to add the flag to the BCR pipeline: bazelbuild/bazel-central-registry#5319

@hofbi
Copy link
Contributor Author

hofbi commented Jul 23, 2025

I started a downstream run at https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/4824

Thanks @fmeum. Do we actually have a baseline for the downstream job? I was looking at a few of the failures showing up and a significant number fails because of Error in rule: rule() got unexpected keyword argument 'incompatible_use_toolchain_transition'. Is this something that is expected to be fixed with #14127?

@fmeum
Copy link
Collaborator

fmeum commented Jul 23, 2025

I don't see any failures that I would attribute to the flip, but it's true that downstream is in bad shape. That makes the signal worse.

Ideally we would be able to run a downstream test with Bazel 8 and the flag flipped instead, which should avoid most of the preexisting failures.

@fweikert Is that possible?

@hofbi
Copy link
Contributor Author

hofbi commented Jul 29, 2025

@fweikert ping

@meteorcloudy
Copy link
Member

I re-opened #6648 and added migration-ready label, we'll see the test result with BCR modules in https://buildkite.com/bazel/bcr-bazel-compatibility-test tomorrow.

But based on historical experience, this one is very hard to flip as many actions do depend on the value of PATH and some other env var.

@hofbi
Copy link
Contributor Author

hofbi commented Aug 11, 2025

Thanks. Let's see what we get tomorrow.

But based on historical experience, this one is very hard to flip as many actions do depend on the value of PATH and some other env var.

Depending how many fixed we would have to do, how about setting --incompatible_strict_action_env=false in projects that heavily depend on it for starters? This would allow us to at list flip the flag here and give projects more time to work this off in their own speed.

@meteorcloudy
Copy link
Member

Depending how many fixed we would have to do, how about setting --incompatible_strict_action_env=false in projects that heavily depend on it for starters?

Yeah, this would work for leaf projects, but if a common dependency actually needs this flag, we'll still need to fix it and migrate the ecosystem first. Let's see the result, I started a manual run at https://buildkite.com/bazel/bcr-bazel-compatibility-test/builds/492

@meteorcloudy
Copy link
Member

Looks like only rules_python might be affected on Windows, can you please take a quick look? If it's not serious, I'm fine with moving this forward and fix new breakages as we discover them.

@hofbi
Copy link
Contributor Author

hofbi commented Aug 11, 2025

Created bazel-contrib/rules_python#3160 to reproduce this on a PR

@hofbi
Copy link
Contributor Author

hofbi commented Aug 28, 2025

@meteorcloudy
Copy link
Member

I think the only difference is that in the rules_python pipeline, it's running again Bazel 7.6.1, however in the BCR pipeline, we are testing against 8.3.1. Can you try upgrade Bazel and see if it's reproducible?

@hofbi hofbi force-pushed the flip-incompatible_strict_action_env branch from fe4238b to 453399e Compare November 12, 2025 14:53
@hofbi hofbi marked this pull request as ready for review November 12, 2025 17:29
@hofbi hofbi requested a review from a team as a code owner November 12, 2025 17:29
@github-actions github-actions bot added team-Remote-Exec Issues and PRs for the Execution (Remote) team awaiting-review PR is awaiting review from an assigned reviewer labels Nov 12, 2025
@hofbi
Copy link
Contributor Author

hofbi commented Nov 12, 2025

bazel-contrib/rules_python#3160 is a fix for rules_python

@meteorcloudy
Copy link
Member

@bazel-io fork 9.0.0

@github-actions github-actions bot removed the awaiting-review PR is awaiting review from an assigned reviewer label Nov 13, 2025
bazel-io pushed a commit to bazel-io/bazel that referenced this pull request Nov 13, 2025
Work towards bazel-contrib/SIG-rules-authors#42.

Fixes bazelbuild#7026

Closes bazelbuild#26587.

PiperOrigin-RevId: 831944371
Change-Id: I397faa463a709b2367d5a6eebc678077c4029bbb
github-merge-queue bot pushed a commit that referenced this pull request Nov 17, 2025
Work towards
bazel-contrib/SIG-rules-authors#42.

Fixes #7026

Closes #26587.

PiperOrigin-RevId: 831944371
Change-Id: I397faa463a709b2367d5a6eebc678077c4029bbb

Commit
e697ff7

Co-authored-by: Markus Hofbauer <[email protected]>
hofbi added a commit to bazel-contrib/bazelrc-preset.bzl that referenced this pull request Dec 16, 2025
rdesgroppes added a commit to rdesgroppes/rules_pkg that referenced this pull request Jan 21, 2026
Bazel 9 flipped `--incompatible_strict_action_env` to `true` by default
(bazelbuild/bazel#26587), which means tests no longer inherit `PATH`
from the host environment. This breaks subprocess calls that rely on
`PATH` lookup:
```
==================== Test output for //distro:packaging_test:
E.
======================================================================
ERROR: testBuild (__main__.PackagingTest.testBuild)
----------------------------------------------------------------------
Traceback (most recent call last):
[...]
    build_result = subprocess.check_output(['bazel', 'build', '--enable_workspace', ':dummy_tar'])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[...]
FileNotFoundError: [Errno 2] No such file or directory: 'bazel'
```

This change therefore adds `env_inherit = ["PATH"]` to that very test,
which matches the behavior of earlier Bazel versions and is consistent
with the presence of a "noci" tag.

Additionally, Bazel 9 removed `WORKSPACE` support entirely
(bazelbuild/bazel#26131), requiring `bzlmod` with `MODULE.bazel`:
```
==================== Test output for //distro:packaging_test:
[...]
WARNING: --enable_bzlmod is set, but no MODULE.bazel file was found at the workspace root. Bazel will create an empty MODULE.bazel file. Please consider migrating your external dependencies from WORKSPACE to MODULE.bazel. For more details, please refer to bazelbuild/bazel#18958.
[...]
ERROR: error loading package '': Unable to find package for @@[unknown repo 'not_named_rules_pkg' requested from @@]//pkg:tar.bzl: The repository '@@[unknown repo 'not_named_rules_pkg' requested from @@]' could not be resolved: No repository visible as '@not_named_rules_pkg' from main repository.
[...]
ERROR: Build did NOT complete successfully
E.
======================================================================
ERROR: testBuild (__main__.PackagingTest.testBuild)
----------------------------------------------------------------------
[...]
    build_result = subprocess.check_output(['bazel', 'build'] + bazel_flags + [':dummy_tar'])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[...]
subprocess.CalledProcessError: Command '['bazel', 'build', '--enable_workspace', ':dummy_tar']' returned non-zero exit status 1.
```

The test now detects the Bazel version at runtime and generates the
appropriate setup:
- Bazel 9+: `MODULE.bazel`, with `module` and `archive_override`,
- earlier versions, as before: `WORKSPACE`, with `http_archive` and
  `--enable_workspace` flag.

For good measure, the change also addresses a leftover TODO by replacing
the `tar` subprocess invocation with pure Python code.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

team-Remote-Exec Issues and PRs for the Execution (Remote) team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bazel 0.21.0 flipped --incompatible_strict_action_env to true and broke itself

3 participants