Skip to content

Conversation

@Wyverald
Copy link
Member

@Wyverald Wyverald requested a review from a team as a code owner August 20, 2025 19:08
@github-actions github-actions bot added team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. awaiting-review PR is awaiting review from an assigned reviewer labels Aug 20, 2025
@Wyverald Wyverald enabled auto-merge August 20, 2025 19:26
@Wyverald Wyverald added this pull request to the merge queue Aug 20, 2025
@iancha1992 iancha1992 added this to the 8.4.0 release blockers milestone Aug 20, 2025
Merged via the queue into release-8.4.0 with commit 6dfb2d6 Aug 20, 2025
49 checks passed
@github-actions github-actions bot removed the awaiting-review PR is awaiting review from an assigned reviewer label Aug 20, 2025
@Wyverald Wyverald deleted the wyv-840-disable-repo-contents-cache branch August 20, 2025 20:12
rdesgroppes added a commit to DataDog/datadog-agent that referenced this pull request Sep 2, 2025
### What does this PR do?
Now `bazelisk` is available on all our CI executors
([macOS](DataDog/ci-platform-machine-images#395)
runners,
[Linux](DataDog/datadog-agent-buildimages#951) &
[Windows](DataDog/datadog-agent-buildimages#953)
containers), this change secures an initial part of our `bazel` setup
while providing reusable job templates to ease caching in CI.

Practically, it adds a handful of jobs focusing on **building `bazel`
dependencies** in the corresponding GitLab `deps_build` stage.

Overall, it consists in:
1. verifying **`bazelisk` properly bootstraps `bazel` across all
platforms**, (primary scope of the PR)
2. extracting initial `bazel:`-prefixed CI job templates and dogfooding
them, (i.e. by building deps)
3. binding `bazelisk`/`bazel` caches to GitLab caching capabilities
(runner-based as of now): installed binaries, "repository cache", "repo
contents cache", and "disk cache" are all saved/restored correctly
to/from GitLab while honoring OS/architecture boundaries,
4. marking in-workspace cache in both `.bazelignore` and `.gitignore`,
5. adjusting code/job ownership accordingly.

### Motivation
Securing some initial `bazel` configuration in CI :
- ensures all platforms are kept in the radar over next iterations,
- prevents regressions over next iterations,
- establishes foundations for future jobs.

### Possible Drawbacks / Trade-offs
- on Windows, compilation errors made be descope dependencies being
built to **only `bzip2`**. This will be of course addressed in a
subsequent PR,
- GitLab caching is still runner-based for the time being, which can be
revisited later. (like everything)

### Additional Notes
Main addressed challenges:
- cache location conflicts[^1]: externalized "repo contents cache"
through `tools/bazel*` wrappers:
  - bazelbuild/bazel#26384
  - bazelbuild/bazel#26522
  - bazelbuild/bazel#26773
  - bazelbuild/bazel#26802
- macOS runners:
  - `bzip2` dependency: fetch from a reachable source:
    - [x] #40219
- Windows:
- `bazel` spawn strategy: fallback to a permissive strategy since
`sandboxed` is unsupported
    - [x] #40328
- `tools/bazel` wrapper: fallback to batch (`tools/bazel.bat`), since
`bash` is discouraged by `bazel` in this case and `tools/bazel.ps1`
poses detection problems,
  - long paths were supported in containers but not on runners:
    - [x] DataDog/ci-platform-machine-images#429
  - recursive symlinks: use `robocopy` instead of `copy`/`move`/`xcopy`.
  
[^1]:
> ERROR: The repo contents cache
[/path/to/datadog-agent/.cache/repo_contents] is inside the workspace
[/path/to/datadog-agent]. This can cause spurious failures. Disable the
repo contents cache with `--repo_contents_cache=`, or specify
`--repo_contents_cache=<path outside the workspace>`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants