-
Notifications
You must be signed in to change notification settings - Fork 4.4k
Distinguish between input and output prefetching in request metadata #25040
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
@bazel-io fork 8.1.0 |
|
@bazel-io fork 7.5.0 |
ea4ad30 added an action as context for prefetches with BwoB, but assumed that all callers would use the prefetcher for inputs to locally executed actions. However, since then, it has also been used to download outputs of completed actions that have been explicitly requested (e.g. as outputs of top-level targets or those matching the regex path patterns). This resulted in downloads with a `prefetcher` action ID and action details for the action that produced the file, rather than consumed it, resulting in confusing situations for observability tools. This is fixed by separately tracking the reason for the fetch and using an action ID of `requested` when the action has the requested file as an output.
742d4d1 to
c4f34e7
Compare
coeuvre
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! LGTM, please fix the tests and I will import it.
Thanks, should be fixed now |
|
During internal review, we changed action id "prefetcher" to "input" and "requested" to "output" because the latter are easier to understand. This is technically a breaking change, but anyone who is consuming this need to update their code anyway. |
ea4ad30 added an action as context for prefetches with BwoB, but assumed that all callers would use the prefetcher for inputs to locally executed actions. However, since then, it has also been used to download outputs of completed actions that have been explicitly requested (e.g. as outputs of top-level targets or those matching the regex path patterns). This resulted in downloads with a `prefetcher` action ID and action details for the action that produced the file, rather than consumed it, resulting in confusing situations for observability tools. This is fixed by separately tracking the reason for the fetch. Using an action ID of `output` when the action has the requested file as an output, and "input" when the action has the requested file as an input. Closes bazelbuild#25040. PiperOrigin-RevId: 719246746 Change-Id: Ib95ff65ba68112b1a38ab3022e5b1a19ef74cc9f (cherry picked from commit 998e762)
…etadata ea4ad30 added an action as context for prefetches with BwoB, but assumed that all callers would use the prefetcher for inputs to locally executed actions. However, since then, it has also been used to download outputs of completed actions that have been explicitly requested (e.g. as outputs of top-level targets or those matching the regex path patterns). This resulted in downloads with a `prefetcher` action ID and action details for the action that produced the file, rather than consumed it, resulting in confusing situations for observability tools. This is fixed by separately tracking the reason for the fetch. Using an action ID of `output` when the action has the requested file as an output, and "input" when the action has the requested file as an input. Closes bazelbuild#25040. PiperOrigin-RevId: 719246746 Change-Id: Ib95ff65ba68112b1a38ab3022e5b1a19ef74cc9f (cherry picked from commit 998e762)
…etadata (#25057) ea4ad30 added an action as context for prefetches with BwoB, but assumed that all callers would use the prefetcher for inputs to locally executed actions. However, since then, it has also been used to download outputs of completed actions that have been explicitly requested (e.g. as outputs of top-level targets or those matching the regex path patterns). This resulted in downloads with a `prefetcher` action ID and action details for the action that produced the file, rather than consumed it, resulting in confusing situations for observability tools. This is fixed by separately tracking the reason for the fetch. Using an action ID of `output` when the action has the requested file as an output, and "input" when the action has the requested file as an input. Closes #25040. PiperOrigin-RevId: 719246746 Change-Id: Ib95ff65ba68112b1a38ab3022e5b1a19ef74cc9f (cherry picked from commit 998e762) RELNOTES: CAS requests made when Bazel downloads a blob with Build without the Bytes enabled now provide metadata with an action ID of `input` if the blob is downloaded as the input to a local action and `output` if it is a requested action output.
#25058) …etadata ea4ad30 added an action as context for prefetches with BwoB, but assumed that all callers would use the prefetcher for inputs to locally executed actions. However, since then, it has also been used to download outputs of completed actions that have been explicitly requested (e.g. as outputs of top-level targets or those matching the regex path patterns). This resulted in downloads with a `prefetcher` action ID and action details for the action that produced the file, rather than consumed it, resulting in confusing situations for observability tools. This is fixed by separately tracking the reason for the fetch. Using an action ID of `output` when the action has the requested file as an output, and "input" when the action has the requested file as an input. Closes #25040. PiperOrigin-RevId: 719246746 Change-Id: Ib95ff65ba68112b1a38ab3022e5b1a19ef74cc9f (cherry picked from commit 998e762) RELNOTES: CAS requests made when Bazel downloads a blob with Build without the Bytes enabled now provide metadata with an action ID of input if the blob is downloaded as the input to a local action and output if it is a requested action output.
|
The changes in this PR have been included in Bazel 7.5.0 RC3. Please test out the release candidate and report any issues as soon as possible. |
ea4ad30 added an action as context for prefetches with BwoB, but assumed that all callers would use the prefetcher for inputs to locally executed actions. However, since then, it has also been used to download outputs of completed actions that have been explicitly requested (e.g. as outputs of top-level targets or those matching the regex path patterns). This resulted in downloads with a `prefetcher` action ID and action details for the action that produced the file, rather than consumed it, resulting in confusing situations for observability tools. This is fixed by separately tracking the reason for the fetch. Using an action ID of `output` when the action has the requested file as an output, and "input" when the action has the requested file as an input. Closes bazelbuild#25040. PiperOrigin-RevId: 719246746 Change-Id: Ib95ff65ba68112b1a38ab3022e5b1a19ef74cc9f
ea4ad30 added an action as context for prefetches with BwoB, but assumed that all callers would use the prefetcher for inputs to locally executed actions. However, since then, it has also been used to download outputs of completed actions that have been explicitly requested (e.g. as outputs of top-level targets or those matching the regex path patterns). This resulted in downloads with a
prefetcheraction ID and action details for the action that produced the file, rather than consumed it, resulting in confusing situations for observability tools.This is fixed by separately tracking the reason for the fetch and using an action ID of
requestedwhen the action has the requested file as an output.