refactor(qa): use transport-native actions in flow scenarios by RomneyDa · Pull Request #97962 · openclaw/openclaw

RomneyDa · 2026-06-30T00:30:03Z

What Problem This Solves

QA flow scenarios injected channel messages by calling low-level bus functions directly. That bypassed the selected transport adapter, so the same scenario could not reliably exercise QA Channel or Crabline through one transport boundary. An attempted universal conversation DSL also added substantial scenario-specific plumbing without matching the existing transport model.

Why This Change Was Made

This replaces the conversation DSL with four YAML-native actions that call QaTransportAdapter directly: resetTransport, sendInbound, waitForOutbound, and waitForNoOutbound.

All 24 direct inbound injections across 15 existing flow scenarios now use sendInbound. Specialized model, command, UI, and thread assertions remain ordinary flow logic instead of expanding the transport interface. The QA Channel and Crabline adapters implement and test the shared actions.

webchat-direct-reply-routing also derives its transcript session key from transport.buildAgentDelivery() instead of hardcoding QA Channel identifiers.

Live Telegram, Slack, Discord, and WhatsApp lanes remain separate channel-specific runners today; this PR does not claim that those runners implement QaTransportAdapter yet.

AI-assisted: yes.

User Impact

No end-user runtime behavior changes. QA authors can write transport-native YAML actions once and run the same inbound flow through QA Channel or Crabline without direct bus calls or a separate conversation language.

The change removes the superseded conversation module and tests. The replacement commit contains 465 additions and 1,919 deletions.

Evidence

End-to-end scenario proof

QA Channel, mock-openai: 14/15 passed. The only excluded result was thinking-slash-model-remap, which correctly requires live-frontier.
QA Channel, live-frontier with openai/gpt-5.5: thinking-slash-model-remap passed. Combined result: all 15 changed scenarios passed on QA Channel using their required provider modes.
Crabline Telegram, mock-openai: all 9 transport-eligible scenarios passed.
Crabline Telegram, live-frontier with openai/gpt-5.5: thinking-slash-model-remap passed. Combined result: all 10 portable changed scenarios passed on Crabline Telegram.
Five scenarios remain intentionally QA-Channel-only via existing requiredChannelDriver: qa-channel: group-message-tool-unavailable-fallback, thread-follow-up, thread-memory-isolation, personal-channel-thread-reply, and control-ui-qa-channel-image-roundtrip.
A Blacksmith Testbox attempt was blocked before provisioning by a GitHub HTTP 429. The completed proof used local real child gateways and Crabline's local Telegram provider.

Artifacts:

.artifacts/qa-e2e/pr-97962-qa-channel/qa-suite-summary.json
.artifacts/qa-e2e/pr-97962-crabline-telegram-eligible/qa-suite-summary.json
.artifacts/qa-e2e/pr-97962-thinking-live-qa-channel/qa-suite-summary.json
.artifacts/qa-e2e/pr-97962-thinking-live-crabline-telegram/qa-suite-summary.json

Automated checks

Full QA Lab suite: 93 test files passed, 1,176 tests passed.
Focused transport and flow suite: 6 test files passed, 55 tests passed.
QA Channel adapter test proves sendInbound, waitForOutbound, and reset through the shared interface.
Crabline Slack adapter test proves sendInbound reaches the fake provider and waitForOutbound observes the provider response through the same interface.
Extension test typecheck passed.
Changed-file oxfmt and oxlint checks passed.
git diff --check passed.
Autoreview completed cleanly with no accepted or actionable findings after both implementation changes.

clawsweeper · 2026-06-30T00:31:00Z

Codex review: needs maintainer review before merge. Reviewed July 1, 2026, 6:30 AM ET / 10:30 UTC.

Summary
The branch adds transport-native QA flow actions, exposes matching QaTransportAdapter methods, and converts selected QA scenario YAML from direct bus calls to the transport boundary.

PR surface: Source +108, Tests +76, Other -25. Total +159 across 24 files.

Reproducibility: not applicable. this is a QA authoring/refactor PR rather than a bug report. The relevant behavior is source-verifiable, and the PR body/comment report the changed scenarios passing through QA Channel and Crabline Telegram.

Review metrics: 2 noteworthy metrics.

QA Flow Actions: 4 added. The new resetTransport, sendInbound, waitForOutbound, and waitForNoOutbound actions are scenario authoring surface after merge.
Scenario Inbound Conversions: 24 conversions across 15 scenarios. Existing flow scenarios move from direct bus injection onto the transport adapter path, which is a meaningful automation blast-radius change.

Stored data model
Persistent data-model change detected: vector/embedding metadata: qa/scenarios/memory/memory-tools-channel-context.yaml, vector/embedding metadata: qa/scenarios/memory/thread-memory-isolation.yaml. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

Maintainer confirms the four YAML actions and adapter methods are the intended QA Lab authoring contract.
[P2] Decide whether to update the QA authoring docs before merge or treat the new action names as internal for now.

Risk before merge

[P1] The four YAML action names and adapter methods become QA Lab authoring/transport contract after merge, so maintainers should explicitly accept that surface.
[P1] The PR rewires existing QA scenarios across the transport path; current-head CI is green and the PR comment reports QA Channel plus Crabline Telegram proof, but the Blacksmith/Testbox attempt was blocked by GitHub HTTP 429.
[P1] The public QA overview currently lists older generic helper names rather than the new YAML-native action names, so documentation follow-through depends on the maintainer's contract decision.

Maintainer options:

Accept The Transport Action Contract (recommended)
Treat resetTransport, sendInbound, waitForOutbound, waitForNoOutbound, and the matching adapter methods as the intended QA Lab authoring surface, then merge with current-head validation.
Document Or Narrow The Authoring Surface
Update the QA authoring docs to describe the new YAML-native actions, or keep any action names internal until maintainers want them as a scenario contract.
Pause For QA Contract Design
Hold this PR if maintainers want the reusable QA transport DSL designed in a separate thread before existing scenarios move onto it.

Next step before merge

[P2] The protected maintainer label and new QA Lab authoring contract require human review; there is no narrow ClawSweeper repair to queue from this read-only review.

Security
Cleared: No concrete security or supply-chain issue was found; the diff is QA Lab TypeScript, tests, and scenario YAML without dependency, workflow, secret, install, or publish changes.

Review details

Best possible solution:

Land this after maintainers accept the QA Lab transport-action contract and decide whether the QA authoring docs should name these YAML-native actions now.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this is a QA authoring/refactor PR rather than a bug report. The relevant behavior is source-verifiable, and the PR body/comment report the changed scenarios passing through QA Channel and Crabline Telegram.

Is this the best way to solve the issue?

Yes, conditionally: moving scenario input/output through the transport adapter is a better owner-boundary fit than direct bus calls or a separate conversation DSL. The remaining question is maintainer acceptance of the new QA Lab action/API contract and docs follow-through.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 4076ba0bd9e8.

Label changes

Label justifications:

P2: This is a normal-priority QA infrastructure improvement with limited end-user impact but meaningful maintainer contract and automation review scope.
merge-risk: 🚨 compatibility: The PR adds QA Lab YAML actions and adapter methods that future scenario authors and transport adapters may treat as supported contract.
merge-risk: 🚨 automation: The PR changes QA scenario runner semantics and rewrites existing scenario flow inputs onto a new transport-native path.
rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: Not applicable for the external-contributor proof gate because this is a repository-member PR; the body and follow-up comment still provide useful QA Channel and Crabline Telegram run evidence.

Evidence reviewed

PR surface:

Source +108, Tests +76, Other -25. Total +159 across 24 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	4	115	7	+108
Tests	5	77	1	+76
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	15	280	305	-25
Total	24	472	313	+159

What I checked:

Repository policy read: Root AGENTS.md was read fully and requires whole-surface PR review plus maintainer handling for compatibility-sensitive plugin/API and automation-contract changes. (AGENTS.md:31, 4076ba0bd9e8)
Scoped extensions policy read: The scoped extensions guide treats bundled plugins as the same boundary third-party plugins see, which makes promoted helper/runtime surfaces contract-sensitive. (extensions/AGENTS.md:1, 4076ba0bd9e8)
Current main still uses direct bus calls: On current main, the DM baseline scenario still calls resetBus, state.addInboundMessage, and waitForOutboundMessage directly, so the central PR behavior is not implemented on main. (qa/scenarios/channels/dm-chat-baseline.yaml:34, 4076ba0bd9e8)
PR head adds adapter methods: The PR head adds reset, sendInbound, waitForNoOutbound, and waitForOutbound to QaTransportAdapter and implements them in the shared state-backed adapter. (extensions/qa-lab/src/qa-transport.ts:182, d584f2346c9e)
PR head adds native flow dispatch: The flow runner dispatches sendInbound, waitForOutbound, waitForNoOutbound, and resetTransport through api.transport, creating the new YAML authoring path. (extensions/qa-lab/src/scenario-flow-runner.ts:164, d584f2346c9e)
PR head converts an existing scenario: The DM baseline scenario on the PR head uses resetTransport, sendInbound, and waitForOutbound while preserving the marker assertion. (qa/scenarios/channels/dm-chat-baseline.yaml:34, d584f2346c9e)

Likely related people:

RomneyDa: Authored merged PR feat(qa): add transport scenario requirements #97915 on transport scenario requirements and is the main recent contributor on the exact QaTransportAdapter/scenario-runtime path this PR extends. (role: recent QA transport scenario contributor; confidence: high; commits: 843ad143647e; files: extensions/qa-lab/src/qa-transport.ts, extensions/qa-lab/src/scenario-runtime-api.ts, extensions/qa-lab/src/qa-transport-scenarios.ts)
mcaxtr: Authored merged PR refactor(qa): add shared QA channel contract and harden worker startup #64562, which introduced qa-transport.ts and qa-channel-transport.ts as the shared contract/base implementation. (role: introduced shared QA transport contract; confidence: high; commits: 000fc7f23346; files: extensions/qa-lab/src/qa-transport.ts, extensions/qa-lab/src/qa-channel-transport.ts)
Peter Steinberger: History shows the scenario-runtime-api.ts surface was introduced in commit fcee268, which is adjacent to the runtime API exposure this PR changes. (role: scenario runtime API contributor; confidence: medium; commits: fcee2683736a; files: extensions/qa-lab/src/scenario-runtime-api.ts)
Gustavo Madeira Santana: History shows suite-runtime-flow.ts was added in commit 4db162d, making this a plausible routing candidate for flow runtime review. (role: QA Lab runtime split contributor; confidence: medium; commits: 4db162db7f3e; files: extensions/qa-lab/src/suite-runtime-flow.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

RomneyDa · 2026-06-30T02:31:45Z

@clawsweeper re-review

clawsweeper · 2026-06-30T02:31:48Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

RomneyDa · 2026-06-30T03:35:29Z

@clawsweeper re-review

clawsweeper · 2026-06-30T03:35:31Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

RomneyDa · 2026-07-01T09:05:59Z

Ran the 15 changed scenarios end to end through both transport paths.

QA Channel: all 15 passed using their required provider modes (14 with mock-openai; thinking-slash-model-remap with live openai/gpt-5.5).
Crabline Telegram: all 10 portable scenarios passed (9 with mock-openai; thinking-slash-model-remap with live openai/gpt-5.5).
The other five scenarios already declare requiredChannelDriver: qa-channel because they require QA-only thread creation, fallback behavior, or Control UI/image semantics.

The initial Crabline run exposed one real portability bug in webchat-direct-reply-routing: its transcript key hardcoded qa-channel. Commit 9a19425049a now derives the route from transport.buildAgentDelivery(), and the scenario passes on both QA Channel and Crabline Telegram.

Blacksmith Testbox was blocked before provisioning by GitHub HTTP 429, so the completed proof used local real child gateways and Crabline's local Telegram provider. Artifact paths are recorded in the PR body.

…w#97962) * Add QA channel scenario driver contracts * Route DM baseline through channel scenario driver * Simplify channel behavior flow runtime helper * test(qa): use native YAML for DM channel scenario * test(qa): add conversation flow action * test(qa): break channel scenario import cycle * test(qa): fix channel scenario lint * Unify QA channel scenario transport adapter * Fix QA transport test type expectations * refactor(qa): use native transport flow actions * test(qa): derive direct reply session from transport

openclaw-barnacle Bot added extensions: qa-lab size: L maintainer Maintainer-authored PR labels Jun 30, 2026

github-actions Bot mentioned this pull request Jun 30, 2026

🦞 OpenClaw 生态日报 2026-06-30 zx0828/big_model_radar#200

Open

openclaw-barnacle Bot added size: XL and removed size: L labels Jun 30, 2026

RomneyDa changed the title ~~test(qa): add channel scenario driver contracts~~ test(qa): route DM baseline through channel scenario driver Jun 30, 2026

RomneyDa force-pushed the dallin/qa-channel-scenario-driver branch from 9a19425 to 80e0773 Compare July 1, 2026 09:26

RomneyDa mentioned this pull request Jul 1, 2026

test: fix stale core test type failures #98551

Merged

RomneyDa force-pushed the dallin/qa-channel-scenario-driver branch from 80e0773 to 8c05211 Compare July 1, 2026 09:59

RomneyDa added 11 commits July 1, 2026 03:02

Add QA channel scenario driver contracts

806d76d

Route DM baseline through channel scenario driver

5de2c24

Simplify channel behavior flow runtime helper

9c0a6f8

test(qa): use native YAML for DM channel scenario

5b1b50e

test(qa): add conversation flow action

fe3d44c

test(qa): break channel scenario import cycle

10afd79

test(qa): fix channel scenario lint

a740a3e

Unify QA channel scenario transport adapter

f491d1b

Fix QA transport test type expectations

6d9fa9f

refactor(qa): use native transport flow actions

9a667b5

test(qa): derive direct reply session from transport

d584f23

RomneyDa force-pushed the dallin/qa-channel-scenario-driver branch from 8c05211 to d584f23 Compare July 1, 2026 10:02

RomneyDa merged commit 24ce672 into main Jul 1, 2026
90 checks passed

RomneyDa deleted the dallin/qa-channel-scenario-driver branch July 1, 2026 18:39

github-actions Bot mentioned this pull request Jul 1, 2026

📡 Upstream Digest — 2026-07-01 19:41 UTC curtismercier/openclaw-mods#1183

Open

clawsweeper Bot mentioned this pull request Jul 1, 2026

test(qa): prove native command targeting across QA transports #98751

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

refactor(qa): use transport-native actions in flow scenarios#97962

refactor(qa): use transport-native actions in flow scenarios#97962
RomneyDa merged 11 commits into
mainfrom
dallin/qa-channel-scenario-driver

RomneyDa commented Jun 30, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jun 30, 2026 •

edited

Loading

Uh oh!

RomneyDa commented Jun 30, 2026

Uh oh!

clawsweeper Bot commented Jun 30, 2026 •

edited

Loading

Uh oh!

RomneyDa commented Jun 30, 2026

Uh oh!

clawsweeper Bot commented Jun 30, 2026 •

edited

Loading

Uh oh!

RomneyDa commented Jul 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

RomneyDa commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

End-to-end scenario proof

Automated checks

Uh oh!

clawsweeper Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RomneyDa commented Jun 30, 2026

Uh oh!

clawsweeper Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RomneyDa commented Jun 30, 2026

Uh oh!

clawsweeper Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RomneyDa commented Jul 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

RomneyDa commented Jun 30, 2026 •

edited

Loading

clawsweeper Bot commented Jun 30, 2026 •

edited

Loading

clawsweeper Bot commented Jun 30, 2026 •

edited

Loading

clawsweeper Bot commented Jun 30, 2026 •

edited

Loading