0.22: extract replay and test into a vetted first-party plugin #3871

Purpose

Ship replay and its lightweight test runner as an optional, first-party plugin in 0.22. Keep agent-device focused on device automation, while preserving explicit .ad workflows and agent-supervised repair for their users. Goal-based test orchestration and replay caching can remain in frameworks such as TesterArmy e2e; this extraction must not replace deterministic .ad execution with model-driven behavior.

Required behavior

• Keep replay, test, replay export, their flags, and the relevant help/MCP/client schemas discoverable without installing executable plugin code. Label optional capabilities as plugins that require separate installation.
• Keep a bundled catalog of vetted capability/package declarations. Use one declaration for help and missing-plugin recovery. The catalog describes capabilities; the runtime registry remains the authority for installed and working implementations. Do not infer an npm package from an unknown command or provider.
• When the plugin is absent, fail before device actions with a typed plugin-not-installed reason, the exact package and installation command, and instructions to retry the original invocation with its arguments intact. For example, the error details include { reason: "plugin-not-installed", package: "<chosen first-party package>", installCommand: "agent-device plugins add <chosen first-party package>" }.
• Distinguish a missing installation from a selected package that fails loading, validation, or setup, and from a running daemon that retains an older plugin set. Preserve typed failure details and give the corresponding repair or restart instructions. Do not install packages or terminate sessions automatically.
• Preserve existing .ad syntax, variable handling, source bundles, target identity verification, divergence reports, resume, session retention, repair transactions, timeouts, cancellation, suite results, reporters, and artifact behavior after explicit installation. Preserve all supported platform routes, including desktop platforms.
• Keep sessions, device dispatch, selector/ref authority, recording authority, policy enforcement, cancellation, and cleanup with their existing core owners. Extract replay execution, suite orchestration, and format engines through explicit host contracts; do not give a plugin parallel ownership of session state.
• Specify the recording boundary: ordinary open/close --save-script capture remains usable in core; replay-specific repair requires the replay plugin. Keep shared .ad data contracts independent of the executable engine.
• Extend the plugin contract for replay capabilities instead of making a replay engine pretend to be a device provider. Integrate the separate replay-backend seam with managed plugin loading and lifecycle ownership. Do not retain a hidden bundled replay engine or silently fall back to a different engine.
• Choose and document the first-party package names, supported core/plugin version range, compatibility checks, and maintainer-owned validation before publication. Define where plugins must be installed when a client uses a remote or embedded daemon; local installation must not imply remote availability.

Observable completion conditions

• A clean core install lists the optional replay/test capability in help without importing its implementation.
• Missing-plugin text and JSON errors give an executable installation command and continuation instructions. CLI, MCP, and typed-client paths preserve the same failure classification.
• After installation, representative existing .ad scripts and suites work without rewriting; repair/resume and Maestro routes preserve their contracts.
• Missing, broken, incompatible, and stale-daemon cases have distinct, tested recovery paths; unknown names do not produce guessed installation commands.
• Core no longer bundles replay execution, suite orchestration, or format engines. Package-closure evidence demonstrates the split without expanding startup imports or adding interaction round trips/captures.
• CI explicitly installs the required first-party plugins. Replay compatibility, provider integration, target-identity guarantees, and device lanes still exercise the extracted implementation.
• CLI help, plugin docs, migration instructions, owning ADRs, and package/layering checks describe the 0.22 boundaries. Existing scripts need installation, not a syntax migration.

Dependencies and sequencing

• Build on the TestMu help and missing-plugin guidance fix linked by its PR.
• Coordinate with Extract the Maestro engine as a plugin (0.22) #3377, which owns Maestro engine extraction and its existing backend seam. Share contracts and package loading rather than introducing competing registries or duplicating its work.
• Coordinate with refactor(provider-webdriver): ship BrowserStack as the @agent-device/browserstack plugin #3309, refactor(provider-webdriver): ship AWS Device Farm as the @agent-device/aws-device-farm plugin #3310, and refactor(provider-limrun): ship Limrun as the @agent-device/limrun plugin #3311 for provider catalog entries and with 0.22: remove superseded plugin, client and environment APIs #3759 for removal of superseded plugin APIs in 0.22.
• Read ADR 0012 for replay identity, divergence, resume, and repair, ADR 0015 for Maestro behavior/performance, ADR 0016/0017 for recording, and ADR 0034 for plugin lifecycle and failure isolation.