Skip to content

test: fold otel smoke into qa e2e#93181

Merged
RomneyDa merged 2 commits into
mainfrom
qa-delete-otel-smoke-script
Jun 17, 2026
Merged

test: fold otel smoke into qa e2e#93181
RomneyDa merged 2 commits into
mainfrom
qa-delete-otel-smoke-script

Conversation

@RomneyDa

@RomneyDa RomneyDa commented Jun 15, 2026

Copy link
Copy Markdown
Member

Summary

  • delete scripts/qa-otel-smoke.ts
  • fold the OTEL smoke runner and receiver assertions into test/e2e/qa-lab/runtime/qa-otel-smoke.e2e.test.ts
  • keep pnpm qa:otel:smoke and pnpm qa:otel:collector-smoke by routing them to the e2e file's --smoke mode
  • remove stale changed-test routing for the deleted script
  • keep qa-otel-smoke coverage metadata scoped to the IDs this scenario actually proves: telemetry.otel and harness.qa-lab

Stacked on #93114.

Verification

  • node scripts/run-vitest.mjs test/e2e/qa-lab/runtime/qa-otel-smoke.e2e.test.ts test/scripts/test-projects.test.ts --reporter=verbose
  • pnpm qa:otel:smoke -- --output-dir .artifacts/qa-e2e/qa-otel-smoke-script-deleted-trimmed
  • pnpm openclaw qa suite --scenario qa-otel-smoke --output-dir .artifacts/qa-e2e/qa-otel-smoke-scenario-script-deleted-trimmed
  • pnpm openclaw qa coverage --match qa-otel-smoke --json
  • pnpm exec oxfmt --check --threads=1 test/e2e/qa-lab/runtime/qa-otel-smoke.e2e.test.ts
  • pnpm exec oxfmt --check --threads=1 qa/scenarios/runtime/qa-otel-smoke.yaml
  • git diff --check

Known local warning during QA commands: existing state migration warning for .openclaw/plugins/installs.json.migrated already being present.

@openclaw-barnacle openclaw-barnacle Bot added scripts Repository scripts size: XL maintainer Maintainer-authored PR labels Jun 15, 2026
@clawsweeper

clawsweeper Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 17, 2026, 4:50 PM ET / 20:50 UTC.

Summary
This PR adds a QA Lab native Vitest scenario for OTEL smoke evidence, moves the existing OTEL smoke test into test/e2e/qa-lab, and routes scripts/qa-otel-smoke.ts edits to that e2e test.

PR surface: Tests +8, Other +29. Total +37 across 5 files.

Reproducibility: not applicable. this is an internal PR review for QA/script cleanup, not a reported bug. The relevant verification is source and CI inspection of the QA scenario path, changed-test routing, and parent stack state.

Review metrics: 2 noteworthy metrics.

  • QA scenario evidence: 1 native Vitest scenario added. The scenario adds scorecard-visible telemetry.otel QA evidence, so maintainers should verify that coverage scope before merge.
  • Changed-test routing: 1 script source route added. Edits to scripts/qa-otel-smoke.ts will select the relocated QA e2e test, so the route must land with the test move.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Settle or retarget the parent PR and rerun the failing CI shards on this exact head before mainline merge.

Risk before merge

  • [P1] The PR is stacked on test: fold lifecycle and package proof into QA Lab #93114 rather than current main, and the current head still shows failing inherited CI shards; maintainers should settle the parent stack before treating this as main-ready.
  • [P1] The PR body still describes older package-script deletion and --smoke routing that are not present in the current 5-file diff, so maintainers should verify the intended scope before merge.

Maintainer options:

  1. Settle the parent stack first (recommended)
    Land, fix, or retarget the open parent QA migration before using this PR as a mainline landing candidate.
  2. Keep this branch stack-only
    Maintainers can merge this into the parent branch if they intentionally preserve the stack and handle the combined CI result there.
  3. Pause if the parent changes direction
    If the broader script-to-QA migration is rewritten or closed, rebuild this OTEL slice against the surviving scenario structure.

Next step before merge

  • [P2] Maintainer sequencing is needed because this protected PR is stacked on an open parent branch with inherited failing CI rather than a narrow automated repair target.

Security
Cleared: No concrete security or supply-chain concern was found; the diff changes QA metadata, test placement, and changed-test routing without touching dependencies, workflows, secrets, permissions, or publishing paths.

Review details

Best possible solution:

Land or retarget this OTEL QA scenario cleanup only after #93114 is green, keeping the scenario metadata, changed-test routing, and surviving OTEL smoke command shape aligned.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this is an internal PR review for QA/script cleanup, not a reported bug. The relevant verification is source and CI inspection of the QA scenario path, changed-test routing, and parent stack state.

Is this the best way to solve the issue?

Yes, with the stack caveat: putting the OTEL smoke assertions behind a native QA Lab Vitest scenario is the right owner boundary, but the parent migration should be green and settled before mainline landing.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 20534c57b7e8.

Label changes

Label justifications:

  • P3: This is low-risk internal QA/script cleanup rather than a user-facing runtime regression.
  • merge-risk: 🚨 other: The PR is stacked on an open parent branch with inherited failing CI, so merge sequencing is not settled by the small diff alone.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The external contributor real-behavior-proof gate does not apply to this member-authored, maintainer-labeled internal QA cleanup; the PR body records targeted commands for maintainer verification.
Evidence reviewed

PR surface:

Tests +8, Other +29. Total +37 across 5 files.

View PR surface stats
Area Files Added Removed Net
Source 0 0 0 0
Tests 3 10 2 +8
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 2 29 0 +29
Total 5 39 2 +37

What I checked:

Likely related people:

  • RomneyDa: Recent merged history shows RomneyDa authored the native QA test-file runner and QA profile/evidence consolidation that this stacked PR extends. (role: recent QA Lab contributor; confidence: high; commits: 561b293c7a96, e8db9c3bc0df, 1f8c4d3958b2; files: extensions/qa-lab/src/test-file-scenario-runner.ts, extensions/qa-lab/src/suite-launch.runtime.ts, qa/scenarios/runtime/qa-otel-smoke.yaml)
  • vincentkoc: Recent merged commits on scripts/qa-otel-smoke.ts tightened the OTEL smoke assertions, receiver cleanup, and collector-output bounds that this PR turns into QA evidence. (role: current OTEL smoke surface contributor; confidence: high; commits: 96980709399a, 4f9f7e20d40c, 08e38464709f; files: scripts/qa-otel-smoke.ts, test/scripts/qa-otel-smoke.test.ts)
  • joshavant: Recent merged OTLP log trace-correlation work touched the same OTEL smoke script and diagnostics behavior covered by the relocated test. (role: recent adjacent contributor; confidence: medium; commits: f3eb8e9714d8; files: scripts/qa-otel-smoke.ts, extensions/diagnostics-otel/src/service.test.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. labels Jun 15, 2026
@RomneyDa
RomneyDa force-pushed the qa-delete-otel-smoke-script branch from 32090f8 to 7897a66 Compare June 15, 2026 06:10
@clawsweeper clawsweeper Bot added the merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. label Jun 15, 2026
@RomneyDa
RomneyDa force-pushed the qa-delete-otel-smoke-script branch from 7897a66 to a63ed23 Compare June 16, 2026 02:24
@RomneyDa
RomneyDa force-pushed the script-to-test-conversion branch from 606494a to 4fa026e Compare June 16, 2026 21:00
@RomneyDa
RomneyDa force-pushed the qa-delete-otel-smoke-script branch from a63ed23 to fd9fc41 Compare June 16, 2026 21:01
@RomneyDa
RomneyDa force-pushed the script-to-test-conversion branch 2 times, most recently from e0571de to 6965711 Compare June 17, 2026 19:56
@RomneyDa
RomneyDa force-pushed the qa-delete-otel-smoke-script branch from fd9fc41 to 771b22a Compare June 17, 2026 20:16
Base automatically changed from script-to-test-conversion to main June 17, 2026 21:22
@openclaw-barnacle openclaw-barnacle Bot added docker Docker and sandbox tooling size: XL and removed size: XS labels Jun 17, 2026
@RomneyDa
RomneyDa force-pushed the qa-delete-otel-smoke-script branch from 771b22a to b784d63 Compare June 17, 2026 21:30
@openclaw-barnacle openclaw-barnacle Bot removed scripts Repository scripts docker Docker and sandbox tooling size: XL labels Jun 17, 2026
@blacksmith-sh

This comment has been minimized.

@RomneyDa
RomneyDa merged commit fae4a01 into main Jun 17, 2026
296 of 302 checks passed
@RomneyDa
RomneyDa deleted the qa-delete-otel-smoke-script branch June 17, 2026 21:55
crh-code pushed a commit to crh-code/openclaw that referenced this pull request Jun 18, 2026
* test: fold otel smoke into qa e2e

* test: eliminate otel smoke script
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jun 18, 2026
* test: fold otel smoke into qa e2e

* test: eliminate otel smoke script
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

extensions: qa-lab maintainer Maintainer-authored PR merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: XS status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant