Skip to content

fix(test): stream QA Lab stdout artifacts#95119

Merged
vincentkoc merged 2 commits into
mainfrom
qa-sre-qalab-runtime-next-20260619
Jun 19, 2026
Merged

fix(test): stream QA Lab stdout artifacts#95119
vincentkoc merged 2 commits into
mainfrom
qa-sre-qalab-runtime-next-20260619

Conversation

@vincentkoc

Copy link
Copy Markdown
Member

Summary

  • stream fallback gateway.stdout.log parsing through the bounded stdout diagnostic capture instead of reading the whole artifact into memory
  • keep stdout diagnostic records discoverable anywhere in the artifact, including before large trailing gateway output
  • add focused QA OTEL smoke coverage for direct artifact streaming and the real fallback path

Verification

  • node --check --experimental-strip-types test/e2e/qa-lab/runtime/qa-otel-smoke-runtime.ts
  • node --check --experimental-strip-types test/e2e/qa-lab/runtime/qa-otel-smoke.e2e.test.ts
  • direct Node import probe for an early diagnostic JSON line followed by a 256 KiB tail
  • git diff --check
  • Fresh autoreview: no actionable findings

node scripts/run-vitest.mjs test/e2e/qa-lab/runtime/qa-otel-smoke.e2e.test.ts was not run locally because this Codex worktree has no node_modules and exits with OPENCLAW_MISSING_VITEST; dependencies were intentionally not installed in the Codex worktree.

@vincentkoc vincentkoc self-assigned this Jun 19, 2026
@openclaw-barnacle openclaw-barnacle Bot added size: S maintainer Maintainer-authored PR labels Jun 19, 2026
@clawsweeper

clawsweeper Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 19, 2026, 7:11 PM ET / 23:11 UTC.

Summary
The branch replaces the QA OTEL smoke fallback artifact read with chunked FileHandle reads and adds tests for stdout diagnostic artifact parsing.

PR surface: Tests +87. Total +87 across 2 files.

Reproducibility: yes. from source, not from a live failing run: current main reads the full fallback gateway.stdout.log with readFile, so the unbounded artifact-read path is clear. I did not execute a large-artifact runtime repro in this read-only review.

Review metrics: 1 noteworthy metric.

  • Fallback artifact read mode: 1 path changed to 64 KiB chunks. The central behavior change bounds memory for the QA stdout artifact fallback before diagnostic JSONL parsing.

Stored data model
Persistent data-model change detected: serialized state: test/e2e/qa-lab/runtime/qa-otel-smoke-runtime.ts, serialized state: test/e2e/qa-lab/runtime/qa-otel-smoke.e2e.test.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🌊 off-meta tidepool
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Run or confirm node scripts/run-vitest.mjs test/e2e/qa-lab/runtime/qa-otel-smoke.e2e.test.ts before marking ready or merging.

Risk before merge

  • [P1] The PR body says the focused node scripts/run-vitest.mjs test/e2e/qa-lab/runtime/qa-otel-smoke.e2e.test.ts run was not executed locally because dependencies were missing, so merge should wait for CI/Testbox or maintainer-run proof.

Maintainer options:

  1. Decide the mitigation before merge
    Land the chunked fallback reader after the maintainer draft is ready and the focused QA OTEL smoke test or equivalent CI/Testbox proof passes.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • The PR is a maintainer-labeled draft with no narrow automated repair needed, so it should stay in normal maintainer review until validation is complete.

Security
Cleared: The diff only touches QA Lab test runtime/test code and does not add dependencies, workflow execution, secret handling, or supply-chain surfaces.

Review details

Best possible solution:

Land the chunked fallback reader after the maintainer draft is ready and the focused QA OTEL smoke test or equivalent CI/Testbox proof passes.

Do we have a high-confidence way to reproduce the issue?

Yes from source, not from a live failing run: current main reads the full fallback gateway.stdout.log with readFile, so the unbounded artifact-read path is clear. I did not execute a large-artifact runtime repro in this read-only review.

Is this the best way to solve the issue?

Yes, this looks like the narrow best fix for the test harness: it preserves the existing stdout diagnostic capture/parser contract and changes only fallback artifact ingestion. Changing the scenario YAML or child stdout listener would not address the whole-file fallback read.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 21c966616f61.

Label changes

Label changes:

  • add P3: This is a low-blast-radius internal QA Lab test-runtime reliability fix, not a user-facing runtime regression.
  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The external-contributor proof gate is not applicable because this is a maintainer-member PR; the PR body lists syntax checks and a direct import probe, while full focused Vitest proof remains a merge validation item.

Label justifications:

  • P3: This is a low-blast-radius internal QA Lab test-runtime reliability fix, not a user-facing runtime regression.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The external-contributor proof gate is not applicable because this is a maintainer-member PR; the PR body lists syntax checks and a direct import probe, while full focused Vitest proof remains a merge validation item.
Evidence reviewed

PR surface:

Tests +87. Total +87 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 0 0 0 0
Tests 2 90 3 +87
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 90 3 +87

What I checked:

Likely related people:

  • vincentkoc: git blame shows the current stdout diagnostic capture and fallback artifact reader in the QA OTEL smoke runtime came from commit 8aa58c5. (role: current implementation author and recent area contributor; confidence: high; commits: 8aa58c5fb0b4; files: test/e2e/qa-lab/runtime/qa-otel-smoke-runtime.ts, test/e2e/qa-lab/runtime/qa-otel-smoke.e2e.test.ts)
  • jesse-merhi: Merged pull request Add stdout diagnostics OTEL log exporter #94561 introduced the diagnostics-otel stdout exporter and related QA smoke scenarios that this PR tightens. (role: adjacent feature owner; confidence: medium; commits: 5db2f6c1fccb; files: extensions/diagnostics-otel/src/service.ts, qa/scenarios/runtime/otel-stdout-log-smoke.yaml, qa/scenarios/runtime/otel-both-log-smoke.yaml)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. labels Jun 19, 2026
@vincentkoc

Copy link
Copy Markdown
Member Author

Land-ready proof for 005cc2a018aeecde9f174d1dc258f428d59699bc:

  • node --check --experimental-strip-types test/e2e/qa-lab/runtime/qa-otel-smoke-runtime.ts
  • node --check --experimental-strip-types test/e2e/qa-lab/runtime/qa-otel-smoke.e2e.test.ts
  • direct Node import probe for an early diagnostic JSON line followed by a 256 KiB tail
  • git diff --check
  • Fresh autoreview: no actionable findings after fixing the initial tail-only false-negative risk
  • Manual exact-head CI release gate 27852391333: success, 127/127 jobs, head SHA 005cc2a018aeecde9f174d1dc258f428d59699bc

Local targeted Vitest was not run in this Codex worktree because node_modules is absent and scripts/run-vitest.mjs exits with OPENCLAW_MISSING_VITEST; dependencies were intentionally not installed in the Codex worktree.

@vincentkoc
vincentkoc marked this pull request as ready for review June 19, 2026 23:16
@vincentkoc
vincentkoc merged commit f9f94e7 into main Jun 19, 2026
207 of 213 checks passed
@vincentkoc
vincentkoc deleted the qa-sre-qalab-runtime-next-20260619 branch June 19, 2026 23:16
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jun 20, 2026
* fix(test): bound QA Lab stdout artifact reads

* fix(test): scan QA Lab stdout artifacts incrementally
lzyyzznl pushed a commit to lzyyzznl/openclaw that referenced this pull request Jun 20, 2026
* fix(test): bound QA Lab stdout artifact reads

* fix(test): scan QA Lab stdout artifacts incrementally
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintainer Maintainer-authored PR P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant