Skip to content

fix(qa-lab): handle stdout/stderr stream errors in gateway child spawns#102104

Merged
steipete merged 1 commit into
openclaw:mainfrom
wangmiao0668000666:fix/qa-lab-gateway-child-stream-errors
Jul 9, 2026
Merged

fix(qa-lab): handle stdout/stderr stream errors in gateway child spawns#102104
steipete merged 1 commit into
openclaw:mainfrom
wangmiao0668000666:fix/qa-lab-gateway-child-stream-errors

Conversation

@wangmiao0668000666

@wangmiao0668000666 wangmiao0668000666 commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

QA Lab spawns short-lived CLI commands and long-lived Gateway children with piped stdout and stderr. A parent-side pipe error had no listener, so Node could raise an uncaught error and crash the QA worker. Merely swallowing that event is also unsafe: the Gateway may keep running after QA can no longer observe its output, and a teardown-only failure could otherwise let a scenario report success.

Why This Change Was Made

This maintainer rewrite uses one first-failure monitor for process, stdout, and stderr errors across every QA Gateway child lifecycle. Short-lived CLI commands reject and stop immediately. Long-lived Gateway children record the failure, log it once, and stop the detached process tree. The active failure getter follows replacement children and is checked at startup, around Gateway operations, and after shutdown cleanup.

The suite cleanup path now runs every cleanup step in order even when the Gateway reports a pipe failure. If both the run and cleanup fail, the aggregate retains the primary suite error as its cause so the existing infrastructure retry policy still recognizes it.

Compared with the submitted head, this removes the duplicated per-stream settlement logic and module-reset spawn mocks, covers the steady-state false-positive case, stops detached process trees rather than only the direct child, and preserves retry classification while cleanup completes.

User Impact

QA runs now fail deterministically when their Gateway output pipes break, without uncaught exceptions, orphaned Gateway trees, false-positive scenario results, skipped cleanup, or lost infrastructure retries. No public CLI, config, plugin, or protocol contract changes.

Evidence

  • Rebased head: d161f73c71b48a7ef7b9aaabd5cb9609eb989862.
  • Crabbox run_19b7cea38d56: Linux Node 24 focused regression suite, 2 files / 102 tests passed after rebasing onto current main.
  • Crabbox run_954ae4389f96: changed-lane extension and test typechecks, lint, database-first and sidecar guards, and runtime import-cycle scan passed.
  • Real child-process coverage destroys the parent-side stdout and stderr readables, verifies CLI rejection and termination, verifies first-failure deduplication for the detached Gateway child, and proves ordered cleanup continues after a resource failure.
  • Fresh autoreview: no accepted or actionable findings; overall correctness 0.91.
  • git diff --check and targeted oxfmt/oxlint passed.

Changelog is not required because this is internal QA infrastructure hardening.

Thanks @wangmiao0668000666 for identifying the missing stream-error handling and providing the original fix direction.

@clawsweeper

clawsweeper Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed July 8, 2026, 11:15 PM ET / 03:15 UTC.

Summary
The branch adds stdout/stderr stream-error handling to QA Lab gateway CLI and child spawns, plus regression tests for the real-spawn and dedup paths.

PR surface: Source +65, Tests +151. Total +216 across 2 files.

Reproducibility: yes. from source inspection: current main has piped QA Lab child streams with data listeners but no stream error listeners, and the PR adds focused tests that exercise that error-event path with real spawned child processes.

Review metrics: none identified.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Refresh or rebase onto current main if maintainers require exact-current-base CI before landing.

Risk before merge

  • [P1] The PR is mergeable but behind current main, so maintainers may require a refresh before landing.

Maintainer options:

  1. Decide the mitigation before merge
    Land the localized QA Lab stream-error fix after ordinary maintainer review because it addresses the current-main gap without adding config, API, dependency, or workflow surface.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • No automated repair is needed; the PR is ready for ordinary maintainer review and any required rebase onto current main.

Security
Cleared: The diff only changes QA Lab child-process stream handling and tests; it adds no dependencies, workflows, permissions, package metadata, generated assets, or secrets handling.

Review details

Best possible solution:

Land the localized QA Lab stream-error fix after ordinary maintainer review because it addresses the current-main gap without adding config, API, dependency, or workflow surface.

Do we have a high-confidence way to reproduce the issue?

Yes, from source inspection: current main has piped QA Lab child streams with data listeners but no stream error listeners, and the PR adds focused tests that exercise that error-event path with real spawned child processes.

Is this the best way to solve the issue?

Yes. The owner-local handler is the narrow maintainable fix for this QA Lab process surface; a broader shared abstraction is not needed for this plugin-only test harness path.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against f77c2aafa095.

Label changes

Label justifications:

  • P2: This is a focused QA Lab process-hardening bug fix with limited blast radius.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes terminal proof from focused tests using the production helper with a real spawned Node child and real pipe-backed stdout/stderr streams, plus mutation-check evidence.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes terminal proof from focused tests using the production helper with a real spawned Node child and real pipe-backed stdout/stderr streams, plus mutation-check evidence.
Evidence reviewed

PR surface:

Source +65, Tests +151. Total +216 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 67 2 +65
Tests 1 151 0 +151
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 218 2 +216

What I checked:

Likely related people:

  • vincentkoc: The live PR is assigned to vincentkoc, and file history shows several QA Lab gateway-child runtime commits by Vincent Koc in the same surface. (role: recent QA Lab gateway contributor and assigned follow-up owner; confidence: high; commits: e085fa1a3f, f312d6c106, e7538b4499; files: extensions/qa-lab/src/gateway-child.ts)
  • steipete: File shortlog/history shows many QA Lab gateway-child stabilization and scenario-support commits by Peter Steinberger. (role: heavy QA Lab area contributor; confidence: medium; commits: 610407730d42, ddfd6c3401, 20266c14cb; files: extensions/qa-lab/src/gateway-child.ts)
  • gumadeiras: Recent QA Lab provider registry, runtime staging, and Matrix lane work touched the same gateway-child file and adjacent QA Lab runtime surface. (role: recent QA Lab feature contributor; confidence: medium; commits: bb7e9823a8, d5933af80b, 653100488d; files: extensions/qa-lab/src/gateway-child.ts, extensions/qa-lab/src)
  • Tran Quang: Current-main blame for the affected helper and spawn lines points to commit 19f7b72, which reintroduced the current gateway-child file snapshot. (role: current-line blame author; confidence: low; commits: 19f7b72a74; files: extensions/qa-lab/src/gateway-child.ts, extensions/qa-lab/src/gateway-child.test.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (11 earlier review cycles; latest 8 shown)
  • reviewed 2026-07-08T08:42:01.622Z sha d3def09 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-08T11:31:45.480Z sha b88ee3d :: needs maintainer review before merge. :: none
  • reviewed 2026-07-08T11:42:39.513Z sha b88ee3d :: needs maintainer review before merge. :: none
  • reviewed 2026-07-09T01:48:12.456Z sha 57605c7 :: needs changes before merge. :: [P1] Accept the gateway log collector here
  • reviewed 2026-07-09T02:16:44.911Z sha 148f3fd :: needs changes before merge. :: [P1] Guard nullable child streams in the helper
  • reviewed 2026-07-09T02:23:52.894Z sha 148f3fd :: needs changes before merge. :: [P1] Narrow the helper to pipe-backed child streams
  • reviewed 2026-07-09T02:38:50.864Z sha 914b9a8 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-09T02:53:21.537Z sha 914b9a8 :: needs maintainer review before merge. :: none

@vincentkoc vincentkoc self-assigned this Jul 8, 2026
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. labels Jul 8, 2026
@wangmiao0668000666

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jul 8, 2026
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jul 9, 2026
@wangmiao0668000666

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Fixed the round-2 P1: typed the helper against the push-only log collector. 148f3fd.

  • gateway-child.ts:400 — attachGatewayChildStreamErrorHandlers now takes instead of
  • mirrors the monitorQaGatewayChildSpawnError contract at line 423
  • runtime behavior unchanged; production call sites already pass createQaGatewayChildLogCollector()

Verified locally on the PR head:

  • pnpm tsgo:extensions → exit 0 (clean cache)
  • node scripts/run-vitest.mjs extensions/qa-lab/src/gateway-child.test.ts → 75/75 pass

🤖 Generated with Claude Code

@clawsweeper

clawsweeper Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Jul 9, 2026
@clawsweeper

clawsweeper Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@steipete
steipete force-pushed the fix/qa-lab-gateway-child-stream-errors branch from 914b9a8 to d161f73 Compare July 9, 2026 09:16
@steipete

steipete commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Land-ready maintainer pass complete at d161f73c71b48a7ef7b9aaabd5cb9609eb989862.

I rewrote this around one first-failure monitor shared by short-lived CLI commands and long-lived/replacement Gateway children. The final shape rejects and terminates CLI children on pipe failure, stops detached Gateway process trees, retains failures across steady-state operations and replacements, completes every ordered suite cleanup step, and preserves the primary infrastructure error as cause so retry classification remains intact.

Proof:

  • Crabbox run_19b7cea38d56: rebased-head Linux Node 24 regression suite, 2 files / 102 tests passed.
  • Crabbox run_954ae4389f96: extension and test typechecks, lint, storage/sidecar guards, and import-cycle checks passed.
  • Fresh autoreview: no accepted/actionable findings; correctness confidence 0.91 after three in-scope lifecycle findings were fixed and re-reviewed.
  • Exact-head hosted CI: run 29007597485, successful.
  • Native review artifacts: validated with zero findings; changelog not required for internal QA infrastructure hardening.

Thanks @wangmiao0668000666 for the original report and fix direction.

@steipete
steipete merged commit 6433b82 into openclaw:main Jul 9, 2026
105 checks passed
@steipete

steipete commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Merged via squash.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

extensions: qa-lab P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: M status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants