Combine official-build artifact uploads for perf#12931
Merged
YuliiaKovalova merged 19 commits intomainfrom Dec 29, 2025
Merged
Conversation
This is supposed to be faster due to a single scan for various 1ES things.
System.IO.DirectoryNotFoundException: Could not find a part of the path 'D:\a\_work\1\s\artifacts\MicroBuild\Output\BootstrapperInfo.json'.
This saves on scanning the final output folder unnecessarily.
…via-template-where-possible
This reverts commit 48c462b.
This reverts commit 4e35b62.
rainersigwald
added
the
Area: Our Own Build
Contributor
There was a problem hiding this comment.
Pull request overview
This PR optimizes official build performance by consolidating artifact uploads to reduce scan overhead. The approach converts individual task-based artifact publishing (using 1ES.PublishBuildArtifacts and related tasks) to declarative templateContext outputs, and introduces a single outputParentDirectory that gets scanned once instead of per-upload. A PowerShell script moves artifacts from various locations into the artifacts/official folder before publishing.
Key changes:
- Converted 7 separate artifact publishing tasks to declarative templateContext outputs
- Introduced outputParentDirectory to enable single-scan optimization
- Added PowerShell script to consolidate artifacts into the official folder before publishing
YuliiaKovalova
approved these changes
Dec 29, 2025
YuliiaKovalova
deleted the
exp/raines/publish-via-template-where-possible
branch
This was referenced Dec 30, 2025
YuliiaKovalova
added a commit
that referenced
this pull request
Jan 14, 2026
## Summary After this change the opt prof publishing stopped working and it causes internal pipeline failures #12931 ## Fix return buildNumber: 'ProfilingInputs/DevDiv/$(Build.Repository.Name)/$(Build.SourceBranchName)/$(Build.BuildNumber)'
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scans injected by policy have dramatically increased our official build time. I consulted with some folks internally, the highest-impact approach we identified was to reduce the number of invocations of the artifact-upload tasks, since each one in the normal flow of the job incurs a scan cost.
To do this we had to:
1ES.PublishBuildArtifactstask calls to declarative templateoutputs:outputParentDirectorythat can be scanned exactly once (for all uploaded artifacts) instead of per-uploadoutputParentDirectoryto contain only things that will be uploaded (artifacts/was too broad and caused the scan to find some false positives).Example successful official build: https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=12895565
(compare that 1h 21m to 2h 34m from a recent
mainbranch official build)Example exp build showing log upload on (artificially injected) failure: https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=12941708