test: add executable runtime fixture canaries#99737
Conversation
|
Codex review: needs maintainer review before merge. Reviewed July 3, 2026, 9:35 PM ET / 01:35 UTC. Summary PR surface: Source +44, Tests +672, Other +72. Total +788 across 14 files. Reproducibility: not applicable. This PR adds executable QA coverage rather than fixing a reported runtime bug. The relevant verification path is the added canary tests and current-head CI/Testbox evidence. Review metrics: none identified. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Risk before merge
Maintainer options:
Next step before merge
Security Review detailsBest possible solution: Keep the PR open for maintainer review and merge once the current-head required checks finish, preserving the executable QA canaries in test/QA-owned surfaces. Do we have a high-confidence way to reproduce the issue? Not applicable: this PR adds executable QA coverage rather than fixing a reported runtime bug. The relevant verification path is the added canary tests and current-head CI/Testbox evidence. Is this the best way to solve the issue? Yes: the coverage is placed at existing owner seams for Voice Call, Gateway/WebChat media, browser Talk, heartbeat scheduling, and QA OTEL evidence without adding production APIs or config. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 0f1fbbe8779d. Label changesLabel changes:
Label justifications:
Evidence reviewedPR surface: Source +44, Tests +672, Other +72. Total +788 across 14 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
|
d671d2c to
c807de8
Compare
* test(qa): add runtime fixture canaries * test(voice-call): track fixture temp directories
What Problem This Solves
Several QA scenario rows described media delivery, browser Talk lifecycle, Voice Call entry points, heartbeat scheduling, and OpenTelemetry behavior without executing the corresponding runtime boundaries. That left the scenario catalog able to report coverage from helper-only or design-level checks rather than reusable executable fixtures.
Why This Change Was Made
This adds bounded canaries at the existing owner seams: mock speech synthesis followed by the real WebChat media-ticket route, a real Playwright page for browser Talk start/stop, mock realtime and Voice Call providers behind actual Gateway/plugin entry points, a real-timer heartbeat schedule check, and direct OTEL producer execution through the shared script evidence writer. The active-run control fixture remains secondary evidence because its embedded-run dependencies are intentionally injected test boundaries.
User Impact
There is no production behavior change. Maintainers get executable, reusable QA evidence for media, Talk, Voice Call, scheduler, and telemetry paths, with evidence claims constrained to the boundaries each fixture actually runs.
Evidence
node scripts/run-vitest.mjs test/e2e/qa-lab/media/webchat-auto-tts.e2e.test.ts test/e2e/qa-lab/voice/active-talk-agent-run-status.e2e.test.ts extensions/voice-call/src/voice-call-cli-rpc-agent-tool.e2e.test.ts src/infra/heartbeat-runner.active-hours-schedule.e2e.test.ts test/e2e/qa-lab/runtime/qa-otel-smoke.e2e.test.ts extensions/qa-lab/src/test-file-scenario-runner.test.ts extensions/qa-lab/src/scenario-catalog.test.ts— 98 assertions passed across the E2E and QA Lab shards.node scripts/run-vitest.mjs run --config test/vitest/vitest.ui-e2e.config.ts --configLoader runner ui/src/ui/e2e/browser-talk-start-stop.e2e.test.ts --reporter=verbose— Playwright Talk lifecycle passed.qa-otel-smoke-runtime.tsexecution — shared evidence statuspass, relative summary artifact resolved correctly, and traces/metrics/correlated logs captured.tbx_01kwnapfrp056k2ndxkmrmx2wr—check:changedpassed typecheck, lint, import-cycle, and repository guard lanes. Actions run: https://github.com/openclaw/openclaw/actions/runs/28690272139