Skip to content

Reland "go_sdk: store SDK filenames and hashes in lockfile facts"#4493

Merged
fmeum merged 3 commits intomasterfrom
reintroduce-facts
Nov 3, 2025
Merged

Reland "go_sdk: store SDK filenames and hashes in lockfile facts"#4493
fmeum merged 3 commits intomasterfrom
reintroduce-facts

Conversation

@fmeum
Copy link
Member

@fmeum fmeum commented Oct 30, 2025

What type of PR is this?

Feature

What does this PR do? Why is it needed?

This moves the download of the "all versions" JSON, which can't hit the repository cache, from each individual go_download_sdk into the module extension. If the current version of Bazel supports facts, this information will also be persisted in the lockfile, allowing for truly airgapped builds assuming an up-to-date download (formerly repository) cache.

This is a reland of #4393 with the following improvements:

  • Adapted to the more limited facts API that actually got merged.
  • Prefetching of SDK hashes is now performed on a best-effort basis so that otherwise airgapped builds that provide SDK hashes do not result in failures.

See bazelbuild/bazel#26198 for the PR that added facts support to Bazel.

Which issues(s) does this PR fix?

Fixes #3945

Other notes for review

You can verify that this works by running bazel mod show_repo @go_default_sdk in the BCR test repo.

…4432)

This reverts commit 6d90ede.

Includes the following fixes:
* Adapted to the more limited facts API that actually got merged.
* Prefetching of SDK hashes is now performed on a best-effort basis so that otherwise airgapped builds that provide SDK hashes do not result in failures.
@fmeum fmeum force-pushed the reintroduce-facts branch 3 times, most recently from fa9681f to d96f699 Compare October 30, 2025 19:27
@fmeum fmeum changed the title Reapply "go_sdk: store SDK filenames and hashes in lockfile facts" Reland "go_sdk: store SDK filenames and hashes in lockfile facts" Oct 30, 2025
@fmeum fmeum marked this pull request as ready for review October 30, 2025 19:30
@fmeum fmeum requested review from dzbarsky and jayconrod October 30, 2025 19:30
@fmeum fmeum force-pushed the reintroduce-facts branch from d96f699 to f8aba64 Compare October 30, 2025 19:34
@fmeum fmeum force-pushed the reintroduce-facts branch from f8aba64 to f79a962 Compare October 30, 2025 19:35
@fmeum fmeum merged commit 30a6f8d into master Nov 3, 2025
4 checks passed
@fmeum fmeum deleted the reintroduce-facts branch November 3, 2025 19:50
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Jan 13, 2026
This upgrade brings several performance improvements and bug fixes:

## Performance improvements (8.5.0)
- Remote execution: Add --remote_max_concurrency_per_connection flag to control concurrent gRPC requests (default: 100)
  bazelbuild/bazel#27564
- Module extensions: Support storing/retrieving JSON-like Starlark objects without invalidation, reducing unnecessary rebuilds
  bazelbuild/bazel#27296

## Reliability improvements (8.5.0)
- Cache invalidation: Source directory contents now tracked for proper invalidation
  bazelbuild/bazel#27417
- Configuration: Add ctx.configuration.short_id for identifying configurations
  bazelbuild/bazel#27128
- Git repositories: git_repository now checks out default branch when unspecified
  bazelbuild/bazel#27705
- Query: Add executables() function and fix genquery for external repos
  bazelbuild/bazel#27560
  bazelbuild/bazel#27117
- Registry mirrors: --module_mirrors now supports per-registry mirror specification
  bazelbuild/bazel#27531

## Bug fixes (8.5.1)
- Remote cache: Add option to continue with local execution if remote cache is unavailable
  bazelbuild/bazel#27996
- Repository handling: Fix crash when mixing use_repo_rule and --inject_repository
  bazelbuild/bazel#27995
- Repository cache: Fix permission denied issue with --experimental_repository_cache_hardlinks
  bazelbuild/bazel#28161
- Local execution: Fix incorrect SkyframeLookupResult usage
  bazelbuild/bazel#27994

Both 8.5.0 and 8.5.1 are fully backward compatible with Bazel 8.0.

## Dependency updates
- Upgrade rules_go from 0.57.0 to 0.59.0 for Bazel 8.5+ compatibility
  bazel-contrib/rules_go#4493
- Configure sh_configure extension for rules_shell to auto-detect shell toolchain

## Platform-specific changes
- Windows: Configure hermetic shell via --repo_env=BAZEL_SH which is used by both
  sh_configure (sh_binary/sh_test) and --shell_executable (genrule/run_shell).
  This eliminates dependency on system environment variables.
- Windows: Disable code coverage collection (--nocollect_code_coverage) to avoid
  shell toolchain issues. Coverage requires sh_binary (collect_coverage) which
  needs a hermetic shell toolchain not yet available.
  bazelbuild/rules_shell#4

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
dd-mergequeue bot pushed a commit to DataDog/datadog-agent that referenced this pull request Jan 15, 2026
### What does this PR do?
Bump `bazel` version from [8.4.2](https://github.com/bazelbuild/bazel/releases/tag/8.4.2) to [8.5.1](https://github.com/bazelbuild/bazel/releases/tag/8.5.1).

To make that happen, we also need to:
- specify which `bash` to use on Windows when evaluating **repository** rules, for compatibility with bazelbuild/bazel#26927:
  ```
  ERROR: /path/to/external/bazel_tools/tools/test/BUILD:23:10: in sh_binary rule @@bazel_tools//tools/test:collect_coverage: 
  Error in fail: No suitable shell toolchain found:
  * if you are running Bazel on Windows, set the BAZEL_SH environment variable to the path of bash.exe
  ```
  (actual job output: https://gitlab.ddbuild.io/DataDog/datadog-agent/-/jobs/1349246422#L77)
- bump `rules_go` from (implicit) [0.57.0](https://github.com/bazel-contrib/rules_go/releases/tag/v0.57.0) to (explicit) [0.59.0](https://github.com/bazel-contrib/rules_go/releases/tag/v0.59.0) for bazel-contrib/rules_go/pull/4493 to be compatible with bazelbuild/bazel/pull/27296:
  ```
  Error: 'Facts' value has no field or method 'clear'
  ```

### Motivation
1. this upgrade alone brings several improvements and bug fixes, among which:
- bazelbuild/bazel#27117
- bazelbuild/bazel#27296
- bazelbuild/bazel#27417, covers:
  - bazelbuild/bazel#25834
  - bazelbuild/bazel#25863
  - bazelbuild/bazel#25864
  - bazelbuild/bazel#25870
  - bazelbuild/bazel#26698
- bazelbuild/bazel#27531
- bazelbuild/bazel#27560
- bazelbuild/bazel#27564
- bazelbuild/bazel#27705
- bazelbuild/bazel#27995
- bazelbuild/bazel#27996

2. `bazel` 9.0 is due soon, so better off favoring incremental bumps.

### Additional Notes
Leveraging [the latter](bazelbuild/bazel#27996) might allow us to later reconsider whether we'd like to go back to the `--remote_cache` flag (instead of the `--remote_executor` flag that we had to switch to in #44962).

Co-authored-by: regis.desgroppes <[email protected]>
JSGette pushed a commit to DataDog/datadog-agent that referenced this pull request Jan 16, 2026
### What does this PR do?
Bump `bazel` version from [8.4.2](https://github.com/bazelbuild/bazel/releases/tag/8.4.2) to [8.5.1](https://github.com/bazelbuild/bazel/releases/tag/8.5.1).

To make that happen, we also need to:
- specify which `bash` to use on Windows when evaluating **repository** rules, for compatibility with bazelbuild/bazel#26927:
  ```
  ERROR: /path/to/external/bazel_tools/tools/test/BUILD:23:10: in sh_binary rule @@bazel_tools//tools/test:collect_coverage: 
  Error in fail: No suitable shell toolchain found:
  * if you are running Bazel on Windows, set the BAZEL_SH environment variable to the path of bash.exe
  ```
  (actual job output: https://gitlab.ddbuild.io/DataDog/datadog-agent/-/jobs/1349246422#L77)
- bump `rules_go` from (implicit) [0.57.0](https://github.com/bazel-contrib/rules_go/releases/tag/v0.57.0) to (explicit) [0.59.0](https://github.com/bazel-contrib/rules_go/releases/tag/v0.59.0) for bazel-contrib/rules_go/pull/4493 to be compatible with bazelbuild/bazel/pull/27296:
  ```
  Error: 'Facts' value has no field or method 'clear'
  ```

### Motivation
1. this upgrade alone brings several improvements and bug fixes, among which:
- bazelbuild/bazel#27117
- bazelbuild/bazel#27296
- bazelbuild/bazel#27417, covers:
  - bazelbuild/bazel#25834
  - bazelbuild/bazel#25863
  - bazelbuild/bazel#25864
  - bazelbuild/bazel#25870
  - bazelbuild/bazel#26698
- bazelbuild/bazel#27531
- bazelbuild/bazel#27560
- bazelbuild/bazel#27564
- bazelbuild/bazel#27705
- bazelbuild/bazel#27995
- bazelbuild/bazel#27996

2. `bazel` 9.0 is due soon, so better off favoring incremental bumps.

### Additional Notes
Leveraging [the latter](bazelbuild/bazel#27996) might allow us to later reconsider whether we'd like to go back to the `--remote_cache` flag (instead of the `--remote_executor` flag that we had to switch to in #44962).

Co-authored-by: regis.desgroppes <[email protected]>
theomagellan pushed a commit to DataDog/datadog-agent that referenced this pull request Jan 19, 2026
### What does this PR do?
Bump `bazel` version from [8.4.2](https://github.com/bazelbuild/bazel/releases/tag/8.4.2) to [8.5.1](https://github.com/bazelbuild/bazel/releases/tag/8.5.1).

To make that happen, we also need to:
- specify which `bash` to use on Windows when evaluating **repository** rules, for compatibility with bazelbuild/bazel#26927:
  ```
  ERROR: /path/to/external/bazel_tools/tools/test/BUILD:23:10: in sh_binary rule @@bazel_tools//tools/test:collect_coverage: 
  Error in fail: No suitable shell toolchain found:
  * if you are running Bazel on Windows, set the BAZEL_SH environment variable to the path of bash.exe
  ```
  (actual job output: https://gitlab.ddbuild.io/DataDog/datadog-agent/-/jobs/1349246422#L77)
- bump `rules_go` from (implicit) [0.57.0](https://github.com/bazel-contrib/rules_go/releases/tag/v0.57.0) to (explicit) [0.59.0](https://github.com/bazel-contrib/rules_go/releases/tag/v0.59.0) for bazel-contrib/rules_go/pull/4493 to be compatible with bazelbuild/bazel/pull/27296:
  ```
  Error: 'Facts' value has no field or method 'clear'
  ```

### Motivation
1. this upgrade alone brings several improvements and bug fixes, among which:
- bazelbuild/bazel#27117
- bazelbuild/bazel#27296
- bazelbuild/bazel#27417, covers:
  - bazelbuild/bazel#25834
  - bazelbuild/bazel#25863
  - bazelbuild/bazel#25864
  - bazelbuild/bazel#25870
  - bazelbuild/bazel#26698
- bazelbuild/bazel#27531
- bazelbuild/bazel#27560
- bazelbuild/bazel#27564
- bazelbuild/bazel#27705
- bazelbuild/bazel#27995
- bazelbuild/bazel#27996

2. `bazel` 9.0 is due soon, so better off favoring incremental bumps.

### Additional Notes
Leveraging [the latter](bazelbuild/bazel#27996) might allow us to later reconsider whether we'd like to go back to the `--remote_cache` flag (instead of the `--remote_executor` flag that we had to switch to in #44962).

Co-authored-by: regis.desgroppes <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants