Skip to content

Fix 40+ second simulator test setup delay#282

Merged
ypopovych merged 13 commits into
mainfrom
fix/diagnose-slow-simulator-setup
Jun 17, 2026
Merged

Fix 40+ second simulator test setup delay#282
ypopovych merged 13 commits into
mainfrom
fix/diagnose-slow-simulator-setup

Conversation

@ypopovych

@ypopovych ypopovych commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Diagnosed and fixed two root causes of a ~50s "Install Test Monitor" delay on iOS/tvOS/watchOS simulators (macOS native was unaffected).

Root cause 1: Environment.init blocking on git fetch (~16s)

DDTestMonitor.env is a lazy static property. Its first access in installTestMonitor() triggered Environment.init(), which synchronously called mergeHeadInfo(). That function runs git fetch --deepen=1 to populate commit author/message metadata. On CI simulators with shallow clones or restricted network access, this fetch times out (~16s).

Fix:

  • Removed mergeHeadInfo from Environment.init()
  • Added Environment.fetchMergeHeadInfo(log:) instance method
  • Added DDTestMonitor.loadGitCommitInfo() which schedules the fetch as the first operation on gitUploadQueue
  • The git upload operation gets a dependency on the merge-head operation, ensuring env.git is fully populated before uploading
  • updateTracerConfig waits for mergeHeadOperation after the settings response (full queue drain if ITR requires git upload, just the merge-head op otherwise)
  • Also moved GitUploader.statusUpToDate (a git status subprocess) into the queue operation so it no longer blocks startup

Root cause 2: GithubCIEnvironmentReader.getJobId scanning huge directory tree (~26s)

GithubCIEnvironmentReader.read() calls getJobId() which expands /**/_diag glob patterns by recursively walking /Users/runner/actions-runner/. On a CI machine this directory tree can be very large, causing FileManager.contentsOfDirectory to iterate thousands of entries.

Fix: Added a maxWildcardDepth=3 cap to expandGlobSegments. The _diag directory is always 1–2 levels deep, so all legitimate paths are still found.

Additional improvements

  • Added Log.measure timing around all waitUntilAllOperationsAreFinished() calls (instrumentationWorkQueue drain, gitUploadQueue drain, testOptimizationSetupQueue drain)
  • Added granular timing inside Environment.init (env-testBundle, env-parallelizationDisabled, env-deviceInfo, env-ciReaders, env-gitInfoAt, etc.)
  • Per-reader timing in the CI reader loop
  • mergeHeadInfo wrapped in Log.measure
  • gitStatusUpToDate wrapped in Log.measure

Test plan

  • Run iOS Simulator tests and verify "Install Test Monitor" drops from ~50s to ~2s
  • Verify commit author/message metadata still appears in test spans (mergeHeadInfo completes before session start via queue drain in setupCrashHandler)
  • Verify ITR git upload still works (upload depends on mergeHead, settings waits for full queue when requireGit)
  • Verify GitHub Actions job ID is still reported correctly (glob depth=3 covers all real runner layouts)
  • Build passes on iOS Simulator, watchOS, macOS

🤖 Generated with Claude Code

ypopovych and others added 10 commits June 15, 2026 17:29
Environment.init() called mergeHeadInfo() synchronously, which runs
`git fetch --deepen=1` on simulators. On CI simulators this fetch
times out (~16s), blocking the entire test setup.

Fix: remove mergeHeadInfo from Environment.init() and schedule it on
instrumentationWorkQueue instead. setupCrashHandler() already waits
for this queue before the session span is created, so the commit
author/message metadata is still available when needed.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…Info

- Wrap all waitUntilAllOperationsAreFinished calls with Log.measure so
  queue drain time is visible in debug output
- Add Environment.fetchMergeHeadInfo(log:) instance method that calls
  mergeHeadInfo and updates self.git, wrapped in Log.measure
- Add DDTestMonitor.loadGitCommitInfo() that schedules fetchMergeHeadInfo
  on instrumentationWorkQueue — called from installTestMonitor, not gated
  by disableSourceLocation (commit metadata is unrelated to source location)

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Log.measure closures inside Environment.init that referenced the stored
property `sourceRoot` caused "self captured before all members initialized".
Introduce a local `sourceRootValue` binding and use it inside the closures.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
getJobId() expands /**/_diag globs by recursively scanning
/Users/runner/actions-runner/, which on a CI machine can be a huge
directory tree. Added a maxWildcardDepth=3 cap to expandGlobSegments
so the ** wildcard stops expanding after 3 levels — the _diag directory
is always 1-2 levels deep, so this preserves full functionality.

Also add per-reader timing to env-ciReaders loop.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
- mergeHeadInfo moves from instrumentationWorkQueue to gitUploadQueue
  (first operation); reference saved in mergeHeadOperation
- gitStatusUpToDate (git subprocess) moves inside the upload operation
  so it no longer blocks test startup
- uploadOp gets a dependency on mergeHeadOperation so env.git is fully
  populated before the upload runs
- updateTracerConfig: when ITR does not require git upload, wait only
  for mergeHeadOperation (not the full queue) to get commit metadata

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
@ypopovych
ypopovych requested review from a team as code owners June 16, 2026 13:15
ypopovych and others added 3 commits June 16, 2026 16:06
Replace the collect-all-then-search approach in getJobId with a
FileManager.searchGlob extension that short-circuits as soon as
the first Worker_ log match is found.

Key changes:
- FileManagerExtensions.swift: new FileManager.searchGlob(_:maxWildcardDepth:body:)
  walks a glob pattern (supporting **) calling body on each matched
  directory; returns true and stops on the first body hit. ** with a
  tail only calls body on dirs matching the full subpath; ** as the
  last segment calls body on every dir in the subtree. Walking starts
  from the longest concrete prefix before the first **, not from /.
- GithubCI.swift: getJobId iterates _diagnosticDirs lazily via
  searchGlob and returns as soon as a match is found, so specific
  dirs (no **) are checked before the broad ** patterns.
- FileManagerExtensionsTests.swift + fixtures/glob/: 12 tests covering
  exact match, missing path, tail vs no-tail body-call semantics,
  early exit, depth capping, and scoped root.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
body returning nil means "keep going"; any non-nil value stops the
walk and is returned directly, eliminating the need for callers to
capture a mutable variable to carry the result.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
@ypopovych
ypopovych requested review from anmarchenko and gnufede June 16, 2026 16:55
@ypopovych
ypopovych merged commit 0ccc4d2 into main Jun 17, 2026
1 check passed
@ypopovych
ypopovych deleted the fix/diagnose-slow-simulator-setup branch June 17, 2026 08:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants