Skip to content

[SDTEST-3792] Rename XCTest retries so Xcode reports each as a distinct test#250

Merged
ypopovych merged 2 commits into
mainfrom
fix/SDTEST-3792-rename-xctest-retries
May 15, 2026
Merged

[SDTEST-3792] Rename XCTest retries so Xcode reports each as a distinct test#250
ypopovych merged 2 commits into
mainfrom
fix/SDTEST-3792-rename-xctest-retries

Conversation

@ypopovych

@ypopovych ypopovych commented May 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Xcode dedupes a real retry failure against the prior run's expected failure when both runs share a test identity. That hid genuine failures emitted on the final attempt for tests routed through ATR/EFD.
  • XCTestCase.addAlias(of:named:) (new, in XCTestExtensions.swift) registers an instance method whose IMP and type encoding are copied from the original test method. Works for sync, throws, async, and methods with return values.
  • DDXCTestRetryGroup.retry() now allocates each retry instance with an alias selector (testFoo$2, testFoo$3, ...). XCTest's invocation-derived -name makes Xcode treat each attempt as a separate test run.
  • DDXCTestRetryGroup.perform(_:) uses the group's captured testId.test (original) for withActiveTest(named:), so the backend keeps a stable test name across retries.
  • Bundles two unrelated drive-by cleanups: drop a stale try in SwiftTestingTrait.prepare(test:) (the call no longer throws) and format elapsed time in Log.measure with %.5f.

Test plan

  • xcodebuild test -only-testing:DatadogSDKTestingTests/DDXCTestObserverTests -only-testing:DatadogSDKTestingTests/SwiftTestingTraitTests — green locally
  • Run an integration test where ATR retries a failing XCTest method; confirm Xcode surfaces every retry's final failure (no dedup against earlier expected failures) and that the backend reports a single test with the original name across attempts
  • Sanity-check an async/throws XCTest retry to confirm the aliased selector dispatches correctly

🤖 Generated with Claude Code

…ct test

Xcode dedupes a real retry failure against the prior run's expected
failure when both runs share a test identity, which hid genuine
failures emitted on the final attempt. Each retry now runs under an
alias selector (testFoo_2, testFoo_3, ...) that forwards to the
original method's IMP, so XCTest's invocation-derived -name makes
Xcode treat every attempt as a separate test. The backend keeps the
original name via the group's captured testId.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@ypopovych
ypopovych requested review from a team as code owners May 14, 2026 14:28
@ypopovych
ypopovych requested a review from calvinbayer May 14, 2026 17:33
@ypopovych
ypopovych merged commit 7274ed6 into main May 15, 2026
16 checks passed
@ypopovych
ypopovych deleted the fix/SDTEST-3792-rename-xctest-retries branch May 15, 2026 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants