Skip to content

Support dynamic inputs with the remote repo contents cache#27634

Open
fmeum wants to merge 1 commit intobazelbuild:masterfrom
fmeum:all-repos-remote
Open

Support dynamic inputs with the remote repo contents cache#27634
fmeum wants to merge 1 commit intobazelbuild:masterfrom
fmeum:all-repos-remote

Conversation

@fmeum
Copy link
Collaborator

@fmeum fmeum commented Nov 11, 2025

With this change, all reproducible repository rules can now be cached in a disk or remote cache, including those with dependencies recorded dynamically during evaluation.

This is made possible by introducing a new intermediate type of synthetic AC entries. When looking up the predeclared inputs hash for a repo rule with dynamic dependencies, the action result for such an intermediate entry lists one or more sets of inputs (e.g. a particular file in another repo or an environment variable name). These inputs are then requested from Skyframe and their current values are hashed to obtain the key of the next AC entry, which is again either an intermediate entry or a final entry containing the contents of the repository.

RELNOTES: The remote repo contents cache now supports all reproducible repo rules.

@fmeum fmeum force-pushed the all-repos-remote branch 4 times, most recently from b4b3182 to 7b4b79f Compare November 17, 2025 11:25
@fmeum fmeum force-pushed the all-repos-remote branch 7 times, most recently from ee70885 to 94c992a Compare November 26, 2025 20:32
@fmeum fmeum changed the title TMP: Support all repo rules with the remote repo contents cache Support dynamic inputs with the remote repo contents cache Dec 16, 2025
@fmeum fmeum force-pushed the all-repos-remote branch 2 times, most recently from 0763010 to fbaafe6 Compare December 16, 2025 11:38
@fmeum fmeum marked this pull request as ready for review December 16, 2025 11:39
@fmeum fmeum requested review from a team, Wyverald, lberki and meteorcloudy as code owners December 16, 2025 11:39
@fmeum fmeum requested review from tjgq and removed request for a team and lberki December 16, 2025 11:39
@github-actions github-actions bot added team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. team-Remote-Exec Issues and PRs for the Execution (Remote) team awaiting-review PR is awaiting review from an assigned reviewer labels Dec 16, 2025
@fmeum
Copy link
Collaborator Author

fmeum commented Dec 16, 2025

@bazel-io fork 9.0.0

Copy link
Member

@Wyverald Wyverald left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comments for the RepoRecordedInputs part; haven't looked at the remote repo cache yet

@fmeum fmeum force-pushed the all-repos-remote branch 4 times, most recently from 1f8fe71 to 5f6d38c Compare January 9, 2026 18:03
@fmeum fmeum force-pushed the all-repos-remote branch 2 times, most recently from 1ea6aa6 to 83f4282 Compare January 20, 2026 15:27
@fmeum fmeum requested review from Wyverald and removed request for meteorcloudy January 20, 2026 16:24
@fmeum
Copy link
Collaborator Author

fmeum commented Jan 20, 2026

@tjgq @Wyverald I rebased onto master, which made the PR much smaller and also allowed me to get rid of the newly introduced Skyframe state. It's now pretty much "just" new business logic.

github-merge-queue bot pushed a commit to bazel-contrib/rules_python that referenced this pull request Feb 23, 2026
This is testing the new API to make use of remote caching mechanisms.

Needs: bazelbuild/bazel#27634

---------

Co-authored-by: Richard Levasseur <[email protected]>
rickeylev added a commit to rickeylev/rules_python that referenced this pull request Feb 23, 2026
…b#3589)

This is testing the new API to make use of remote caching mechanisms.

Needs: bazelbuild/bazel#27634

---------

Co-authored-by: Richard Levasseur <[email protected]>
(cherry picked from commit 06aa36d)
rickeylev added a commit to bazel-contrib/rules_python that referenced this pull request Feb 23, 2026
This is testing the new API to make use of remote caching mechanisms.

Needs: bazelbuild/bazel#27634

---------

(cherry picked from commit 06aa36d)

Co-authored-by: Ignas Anikevicius <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting-review PR is awaiting review from an assigned reviewer team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. team-Remote-Exec Issues and PRs for the Execution (Remote) team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants