[7.2.0] Fix a potential infinite loop in the case of an interruption. #22647
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The Issue
Some external users reported the following sequence:
What happened under the hood:
The issue could be reproduced very reliably by placing a breakpoint here[1] and interrupt the build.
Bazel is in the middle of the recursive
IncrementalPackageRoots.registerAndPlantMissingSymlinksmethod when it received the interruption.One important detail: we only add a NestedSet to the
donePackagesRefset when the method is done successfully. When there's an interruption, we always bail early and never actually reach this line where the NestedSet is added to the set[2].Without deduplication, this could lead to what feels like an finite loop if the packages are structured like so:
In this case, NestedSet
[A]represents a common child of many NestedSets and would be repeated again and again. We've indeed observed this in a real build, making it unable to finish within any reasonable timeframe.The Solution
It was overly restrictive to only commit a NestedSet into the de-dup set after all of its symlinks have been planted. It only makes sense if we're planting the symlinks for multiple top-level targets at the same time and want to avoid the situation where a top-level target is allowed to enter execution without all of its symlinks planted. We're already avoiding this situation by design by planting the symlinks for 1 single top-level target at a time.
To avoid the near-infinite loop caused by a repeated NestedSet, we add each NestedSet to the de-duplication set the very first time it's seen.
Changes in this CL
Future.get()instead of 1 for each recursive layer.Fixes #22586.
[1]
bazel/src/main/java/com/google/devtools/build/lib/skyframe/IncrementalPackageRoots.java
Line 253 in 193b114
[2]
bazel/src/main/java/com/google/devtools/build/lib/skyframe/IncrementalPackageRoots.java
Line 256 in 193b114
PiperOrigin-RevId: 640524271
Change-Id: I63c39d7c8f27abaf9229396af1424e775cf5f85f
Commit d705928