E2E Test Report: PARTIAL VERIFICATION ⚠️

Tested by: Shepherd Agent (autonomous QA for Datadog Test Optimization)

Test Environment

• Method: Local testing
• Playground: SwiftLint (XCTest, 892 test methods, 362 suites)
• Revision tested: 5c87d693f1eb8fab9472d58b4b2843c50be2bdf4
• Backend: mockdog with itr-enabled scenario (code_coverage: true, tests_skipping: true, itr_enabled: true)

Mockdog capture

• 1 session, 1 module, 362 suites, 1040 tests
• 1037 pass, 2 skip, 1 fail (the failure is an unrelated SwiftLint fixture, not from the SDK change)
• The xctest helper crashed with signal 6 at the very end (swift-testing helper crash, unrelated to this PR). All spans were exported before the crash.

test.itr.tests_skipping.enabled propagation

Suite-level propagation works — every suite span carries test.itr.tests_skipping.enabled=true via the new `testSuiteWillEnd` hook at `Sources/DatadogSDKTesting/TestImpactAnalysis/TestImpactAnalysis.swift:93-98`. ✅

Test-level propagation does NOT appear on test spans in real XCTest export, despite the new `test.set(tag: ..., value: isSkippingEnabled)` call at `TestImpactAnalysis.swift:127`. ❌

Likely cause

The code paths for ${\backslash}set(tag:value:)$ differ between `Suite` and `Test`:

• `Suite.set(tag:value:)` (`DDTestSuite.swift:133`) writes directly via `value.spanAttribute` — works.
• `Test.set(tag:value:)` (`DDTest.swift:121`) routes through `setTag(key:value:Any) → span.setAttribute(key:, value: AttributeValue(value))`. The OpenTelemetry `AttributeValue.init?(_:Any)` is failable; when the value arrives boxed as `any SpanAttributeConvertible` wrapping a `Bool`, the `case let val as Bool` pattern may not match through the existential, the failable init returns `nil`, and the attribute is silently dropped.

Supporting evidence:

• `test.skip_reason` (a `String`) set via the same `set(tag:)` call DOES land on test spans (2 spans in this run).
• `test.is_ui_test` is set as the literal string `"true"`/`"false"`, not as a `Bool` (see `DDTest.swift:28`).
• The PR's unit tests pass because `Mocks.Runner` records tags directly without routing through `AttributeValue.init?(Any)`, so the Bool path is exercised only in real XCTest.

Suggested fix

Either:

1. Pass the value as a string at line 127 (consistent with `test.is_ui_test`):
   ```swift
   test.set(tag: DDTestSessionTags.testSkippingEnabled, value: isSkippingEnabled ? "true" : "false")
   ```
2. Or align `Test.set(tag:value:)` with the suite implementation, using `value.spanAttribute` directly so the protocol existential is unboxed.

Test methodology

1. Cloned the PR branch into the SwiftLint playground via `crook --dep dd-sdk-swift-testing=anmarchenko/tests_skipped_tag_propagation`.
2. Built the SDK XCFramework from source, then ran SwiftLint's XCTest suite (892 tests) with auto-instrumentation.
3. Captured all CI Visibility events into mockdog (`itr-enabled` scenario).
4. Queried mockdog's parsed JSONL spans (`tests.jsonl`, `test_suites.jsonl`, `test_sessions.jsonl`) for the new tag.

This E2E test was performed by Shepherd — autonomous QA agent for Datadog Test Optimization.

Fixed the e2e failure in 485f3c2. The internal TestRun.set(tag:value:) path now writes value.spanAttribute directly to the OpenTelemetry span, so Bool values are emitted as the expected string attributes ("true"/"false") instead of being dropped through the Any conversion path.

Added a real-span regression test covering Bool tag conversion on test spans.

Verified with:
xcodebuild test -project DatadogSDKTesting.xcodeproj -scheme DatadogSDKTesting -destination platform=macOS -derivedDataPath /private/tmp/dd-sdk-swift-testing-DerivedData-fresh

Result: 356 XCTest tests passed, plus the Swift Testing run passed with known issues only.