Skip to content

Extend progressive cache to bazel-managed Go and pip#47007

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 2 commits intomainfrom
regis.desgroppes/xdg-cache-for-go-and-pip
Feb 27, 2026
Merged

Extend progressive cache to bazel-managed Go and pip#47007
gh-worker-dd-mergequeue-cf854d[bot] merged 2 commits intomainfrom
regis.desgroppes/xdg-cache-for-go-and-pip

Conversation

@rdesgroppes
Copy link
Contributor

@rdesgroppes rdesgroppes commented Feb 26, 2026

What does this PR do?

Wire up bazel-managed Go and pip caches to XDG_CACHE_HOME in the tools/bazel* scripts, and add the corresponding paths to the progressive GitLab runner cache keyed on .go-version and .python-version.

Motivation

Extend #43274's XDG-as-single-cache-root design to Go and pip, whose XDG support has been steadlessly growing from "Partial" to "Supported": https://wiki.archlinux.org/title/XDG_Base_Directory.

By landing in $XDG_CACHE_HOME, they inherit the progressive-cache policy from #46151: only main pushes to them, keeping growth bounded.
Keying on language version files further contains growth by resetting the cache at version upgrade boundaries rather than accumulating superseded artifacts.

Additional Notes

No worry: the new cache paths are already excluded from omnibus source trees via **/.cache/**/* source filters.

Coming soon: we might want to leverage upcoming --strict_repo_env (with bazel 8.6.0, #47011), for which we'll anyway have to list propagated environment variables.

Near future: as the omnibus-bazel transition progresses, other caches (cache_omnibus_ruby_deps, go_deps, go_tools_deps, go_tools_deps_arm64, etc.) are expected to shrink until no longer applicable.

@rdesgroppes rdesgroppes added changelog/no-changelog No changelog entry needed qa/no-code-change No code change in Agent code requiring validation labels Feb 26, 2026
@dd-octo-sts dd-octo-sts bot added internal Identify a non-fork PR team/agent-build labels Feb 26, 2026
@github-actions github-actions bot added the medium review PR review might take time label Feb 26, 2026
@agent-platform-auto-pr
Copy link
Contributor

agent-platform-auto-pr bot commented Feb 26, 2026

Gitlab CI Configuration Changes

⚠️ Diff too large to display on Github.

Changes Summary

Removed Modified Added Renamed
0 110 0 0

ℹ️ Diff available in the job log.

@agent-platform-auto-pr
Copy link
Contributor

agent-platform-auto-pr bot commented Feb 26, 2026

Files inventory check summary

File checks results against ancestor f8d99651:

Results for datadog-agent_7.78.0~devel.git.182.2964ea7.pipeline.99454997-1_amd64.deb:

Detected file changes:

38 Added files:

  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.0.dist-info/RECORD (14.81 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.0.dist-info/licenses/LICENSE (11.61 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/msal-1.34.0.dist-info/METADATA (11.35 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.6.0.dist-info/LICENSE (11.09 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.0.dist-info/licenses/AUTHORS (8.96 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.6.0.dist-info/RECORD (3.7 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/msal-1.34.0.dist-info/RECORD (3.63 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.0.dist-info/METADATA (3.03 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.6.0.dist-info/AUTHORS (2.75 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/certifi-2026.1.4.dist-info/METADATA (2.42 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.6.0.dist-info/METADATA (2.22 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.0.dist-info/entry_points.txt (1.81 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/certifi-2026.1.4.dist-info/RECORD (1.18 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/msal-1.34.0.dist-info/licenses/LICENSE (1.12 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/certifi-2026.1.4.dist-info/licenses/LICENSE (989.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.6.0.dist-info/entry_points.txt (387.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.0.dist-info/direct_url.json (315.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.6.0.dist-info/direct_url.json (306.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/certifi-2026.1.4.dist-info/direct_url.json (293.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/msal-1.34.0.dist-info/direct_url.json (285.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/certifi-2026.1.4.dist-info/WHEEL (91.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.0.dist-info/WHEEL (91.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/msal-1.34.0.dist-info/WHEEL (91.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.6.0.dist-info/WHEEL (91.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.0.dist-info/pbr.json (46.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.6.0.dist-info/pbr.json (46.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.0.dist-info/top_level.txt (14.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.6.0.dist-info/top_level.txt (10.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/certifi-2026.1.4.dist-info/top_level.txt (8.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/msal-1.34.0.dist-info/top_level.txt (5.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/certifi-2026.1.4.dist-info/INSTALLER (4.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.0.dist-info/INSTALLER (4.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/msal-1.34.0.dist-info/INSTALLER (4.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.6.0.dist-info/INSTALLER (4.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/certifi-2026.1.4.dist-info/REQUESTED (0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.0.dist-info/REQUESTED (0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/msal-1.34.0.dist-info/REQUESTED (0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.6.0.dist-info/REQUESTED (0 B)

81 Removed files:

  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click/core.py (129.01 KiB)
  • opt/datadog-agent/LICENSES/krb5-NOTICE (62.62 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click/types.py (38.99 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click/termui.py (30.31 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click/_termui_impl.py (26.46 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click/shell_completion.py (20.5 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click/utils.py (19.78 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click/testing.py (18.65 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click/parser.py (18.56 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click/_compat.py (18.25 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click/decorators.py (18.03 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.1.dist-info/RECORD (14.81 KiB)
  • opt/datadog-agent/LICENSES/msodbcsql18-LICENSE.txt (11.85 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.1.dist-info/licenses/LICENSE (11.61 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/aws_msk_iam_sasl_signer_python-1.0.2.dist-info/METADATA (11.48 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/msal-1.35.0.dist-info/METADATA (11.4 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.7.0.dist-info/licenses/LICENSE (11.09 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/aws_msk_iam_sasl_signer_python-1.0.2.dist-info/LICENSE (11.09 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click/exceptions.py (9.72 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click/formatting.py (9.5 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/aws_msk_iam_sasl_signer/MSKAuthTokenProvider.py (9.29 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.1.dist-info/licenses/AUTHORS (8.99 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click/_winconsole.py (8.27 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/datadog_checks/base/utils/http_testing.py (4.52 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click/__init__.py (4.37 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.7.0.dist-info/RECORD (3.72 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/msal-1.35.0.dist-info/RECORD (3.63 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.1.dist-info/METADATA (2.85 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.7.0.dist-info/licenses/AUTHORS (2.75 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click-8.3.1.dist-info/RECORD (2.65 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click-8.3.1.dist-info/METADATA (2.56 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/certifi-2026.2.25.dist-info/METADATA (2.42 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.7.0.dist-info/METADATA (2.32 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click/globals.py (1.88 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.1.dist-info/entry_points.txt (1.81 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/aws_msk_iam_sasl_signer/cli.py (1.81 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/aws_msk_iam_sasl_signer_python-1.0.2.dist-info/RECORD (1.51 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/datadog_checks/base/utils/http_protocol.py (1.49 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click-8.3.1.dist-info/licenses/LICENSE.txt (1.44 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click/_textwrap.py (1.37 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/certifi-2026.2.25.dist-info/RECORD (1.19 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/msal-1.35.0.dist-info/licenses/LICENSE (1.12 KiB)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/certifi-2026.2.25.dist-info/licenses/LICENSE (989.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click/_utils.py (943.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/datadog_checks/base/utils/http_exceptions.py (749.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.7.0.dist-info/entry_points.txt (387.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/aws_msk_iam_sasl_signer_python-1.0.2.dist-info/direct_url.json (340.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.1.dist-info/direct_url.json (315.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.7.0.dist-info/direct_url.json (306.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/certifi-2026.2.25.dist-info/direct_url.json (294.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click-8.3.1.dist-info/direct_url.json (286.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/msal-1.35.0.dist-info/direct_url.json (285.0 B)
  • opt/datadog-agent/embedded/bin/aws_msk_get_auth_token (228.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/aws_msk_iam_sasl_signer/__init__.py (194.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/aws_msk_iam_sasl_signer_python-1.0.2.dist-info/WHEEL (109.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/certifi-2026.2.25.dist-info/WHEEL (91.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.1.dist-info/WHEEL (91.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/msal-1.35.0.dist-info/WHEEL (91.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.7.0.dist-info/WHEEL (91.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click-8.3.1.dist-info/WHEEL (82.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/aws_msk_iam_sasl_signer_python-1.0.2.dist-info/entry_points.txt (79.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.1.dist-info/pbr.json (46.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.7.0.dist-info/pbr.json (46.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/aws_msk_iam_sasl_signer_python-1.0.2.dist-info/top_level.txt (24.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.1.dist-info/top_level.txt (14.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.7.0.dist-info/top_level.txt (10.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/certifi-2026.2.25.dist-info/top_level.txt (8.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/msal-1.35.0.dist-info/top_level.txt (5.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/aws_msk_iam_sasl_signer_python-1.0.2.dist-info/INSTALLER (4.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/certifi-2026.2.25.dist-info/INSTALLER (4.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click-8.3.1.dist-info/INSTALLER (4.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.1.dist-info/INSTALLER (4.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/msal-1.35.0.dist-info/INSTALLER (4.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.7.0.dist-info/INSTALLER (4.0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/aws_msk_iam_sasl_signer_python-1.0.2.dist-info/REQUESTED (0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/certifi-2026.2.25.dist-info/REQUESTED (0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click/py.typed (0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/click-8.3.1.dist-info/REQUESTED (0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/keystoneauth1-5.13.1.dist-info/REQUESTED (0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/msal-1.35.0.dist-info/REQUESTED (0 B)
  • opt/datadog-agent/embedded/lib/python3.13/site-packages/stevedore-5.7.0.dist-info/REQUESTED (0 B)

@rdesgroppes rdesgroppes force-pushed the regis.desgroppes/xdg-cache-for-go-and-pip branch from 8c2b505 to 34eaa5c Compare February 27, 2026 07:34
@github-actions github-actions bot added short review PR is simple enough to be reviewed quickly and removed medium review PR review might take time labels Feb 27, 2026
@rdesgroppes rdesgroppes force-pushed the regis.desgroppes/xdg-cache-for-go-and-pip branch from 34eaa5c to 45e5956 Compare February 27, 2026 08:02
@rdesgroppes rdesgroppes marked this pull request as ready for review February 27, 2026 08:02
@rdesgroppes rdesgroppes requested a review from a team as a code owner February 27, 2026 08:02
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 45e5956b29

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@github-actions github-actions bot added medium review PR review might take time and removed short review PR is simple enough to be reviewed quickly labels Feb 27, 2026
@rdesgroppes rdesgroppes force-pushed the regis.desgroppes/xdg-cache-for-go-and-pip branch from 45e5956 to 23e4808 Compare February 27, 2026 13:19
### What does this PR do?
Wire up `bazel`-managed Go and `pip` caches to `XDG_CACHE_HOME` in the
`tools/bazel*` scripts, and add the corresponding paths to the
progressive GitLab runner cache keyed on `.go-version` and
`.python-version`.

### Motivation
Extend #43274's XDG-as-single-cache-root design to Go and `pip`, whose
XDG support has been steadlessly growing from "Partial" to "Supported":
https://wiki.archlinux.org/title/XDG_Base_Directory.

By landing in `$XDG_CACHE_HOME`, they inherit the progressive-cache
policy from #46151: only `main` pushes to them, keeping growth bounded.
Keying on language version files further contains growth by resetting
the cache at version upgrade boundaries rather than accumulating
superseded artifacts.

### Additional Notes
No worry: the new cache paths are already excluded from omnibus source
trees via `**/.cache/**/*` source filters.

Coming soon: we might want to leverage upcoming `--strict_repo_env`
(with `bazel` 8.6+), for which we'll anyway have to list allowed
environment variables.

Near future: as the omnibus-bazel transition progresses, other caches
(`cache_omnibus_ruby_deps`, `go_deps`, `go_tools_deps`,
`go_tools_deps_arm64`, etc.) are expected to shrink until no longer
applicable.
Fail early with an informative error if XDG_CACHE_HOME is set but does
not denote an absolute path.

A relative XDG_CACHE_HOME would silently produce distinct cache
locations depending on the working directory, potentially mixing
artifacts across workspaces or failing to locate cached files.
@rdesgroppes rdesgroppes force-pushed the regis.desgroppes/xdg-cache-for-go-and-pip branch from 23e4808 to 2964ea7 Compare February 27, 2026 15:43
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d bot merged commit 16eda26 into main Feb 27, 2026
258 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d bot deleted the regis.desgroppes/xdg-cache-for-go-and-pip branch February 27, 2026 18:43
@github-actions github-actions bot added this to the 7.78.0 milestone Feb 27, 2026
rdesgroppes added a commit that referenced this pull request Feb 27, 2026
rdesgroppes added a commit that referenced this pull request Feb 27, 2026
rdesgroppes added a commit that referenced this pull request Feb 27, 2026
rdesgroppes added a commit that referenced this pull request Feb 28, 2026
)" (#47083)

### What does this PR do?
This reverts commit 16eda26.

### Motivation
#incident-50242.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/no-changelog No changelog entry needed internal Identify a non-fork PR medium review PR review might take time qa/no-code-change No code change in Agent code requiring validation team/agent-build team/agent-devx team/ebpf-platform

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants