Skip to content

test(perf): compare saved CLI startup benchmarks#94812

Merged
clawsweeper[bot] merged 5 commits into
mainfrom
clawsweeper/automerge-openclaw-openclaw-85400
Jun 19, 2026
Merged

test(perf): compare saved CLI startup benchmarks#94812
clawsweeper[bot] merged 5 commits into
mainfrom
clawsweeper/automerge-openclaw-openclaw-85400

Conversation

@clawsweeper

@clawsweeper clawsweeper Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Makes #85400 merge-ready for the ClawSweeper automerge loop.
The edit pass should inspect the live PR diff, review comments, and failing checks; rebase if needed; keep the contributor branch credited; and stop only when validation is green or an external blocker is proven.

Source

Replacement for #85400 because the source branch could not be updated.

Original contributor: @FelixIsaac.

@clawsweeper clawsweeper Bot added scripts Repository scripts clawsweeper:automerge Maintainer opted this PR into bounded ClawSweeper-reviewed automerge clownfish:automerge Maintainer opted this Clownfish PR into bounded ClawSweeper-reviewed automerge clawsweeper Tracked by ClawSweeper automation labels Jun 19, 2026
@clawsweeper

clawsweeper Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

Codex review: passed. Reviewed June 19, 2026, 5:36 AM ET / 09:36 UTC.

Summary
Adds saved CLI startup benchmark report comparison flags to scripts/bench-cli-startup.ts, plus JSON output coverage and changed-target routing expectations for the new test-helper importer.

PR surface: Tests +77, Other +109. Total +186 across 4 files.

Reproducibility: not applicable. as a feature/tooling PR. The prior PR defects were source-proven in review comments and the current head addresses them; I did not run local tests because this review was read-only.

Review metrics: 1 noteworthy metric.

  • Script CLI options: 2 added, 0 removed. The benchmark harness gains saved-report comparison flags, so reviewers should notice the new local tooling contract before merge.

Root-cause cluster
Relationship: canonical
Canonical: #94812
Summary: This PR is the canonical open replacement for the closed unmerged saved-report comparison source PR.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🌊 off-meta tidepool
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • none.

Next step before merge

  • [P2] No repair job is needed because this automerge-lane PR has no remaining narrow code defect from this review.

Security
Cleared: The diff only reads caller-provided local benchmark JSON and writes local JSON output, with no dependency, workflow, permission, secret, or remote-code-execution changes.

Review details

Best possible solution:

Keep the narrow benchmark-harness compare mode and let the existing exact-head automerge/check gates decide the PR.

Do we have a high-confidence way to reproduce the issue?

Not applicable as a feature/tooling PR. The prior PR defects were source-proven in review comments and the current head addresses them; I did not run local tests because this review was read-only.

Is this the best way to solve the issue?

Yes. The benchmark script already owns saved benchmark JSON output, and adding comparison there with focused output and routing coverage is the narrow maintainable path.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against a8b5f5d5518b.

Label changes

Label changes:

  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • remove rating: 🦪 silver shellfish: Current PR rating is rating: 🐚 platinum hermit, so this older rating label is no longer current.

Label justifications:

  • P3: This is a low-risk repository benchmark-harness improvement with no production runtime blast radius.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • status: 🚀 automerge armed: This PR is in ClawSweeper's automerge lane. Not applicable: This is a ClawSweeper bot replacement PR, so the external-contributor proof gate does not apply directly; the linked source PR supplied copied live terminal output for the central compare-mode behavior.
Evidence reviewed

PR surface:

Tests +77, Other +109. Total +186 across 4 files.

View PR surface stats
Area Files Added Removed Net
Source 0 0 0 0
Tests 3 78 1 +77
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 1 123 14 +109
Total 4 201 15 +186

What I checked:

Likely related people:

  • vincentkoc: Recent GitHub path history repeatedly touches scripts/bench-cli-startup.ts and its tests, and a maintainer command armed the original source PR for automerge. (role: recent benchmark harness contributor; confidence: high; commits: 77f8b1671629, 59365959d3b0; files: scripts/bench-cli-startup.ts, test/scripts/bench-cli-startup.test.ts)
  • RomneyDa: Recent commits updated temp-dir helper routing expectations in the same changed-target test surface affected by this PR. (role: recent changed-test routing contributor; confidence: medium; commits: dfd8a2220b92; files: src/scripts/test-projects.test.ts, test/scripts/test-projects.test.ts)
  • hxy91819: Recent history introduced the shared temp-dir helper guidance and helper surface that the new benchmark test imports. (role: shared test-helper contributor; confidence: medium; commits: 06431fd99b9e; files: test/helpers/temp-dir.ts, test/helpers/AGENTS.md)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 🚀 automerge armed This PR is in ClawSweeper's automerge lane. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. labels Jun 19, 2026
@clawsweeper

clawsweeper Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

🦞✅
ClawSweeper merged this PR after the passing review.

Source: clawsweeper[bot]
Feedback: structured ClawSweeper verdict: pass (sha=1afa110f1bc86beb77fd434b3edd2c75cfb33c17)
Merge status: merged by ClawSweeper automerge
Merged at: 2026-06-19T09:37:48Z
Merge commit: 2e0dfda46219

What merged:

  • Adds saved CLI startup benchmark report comparison flags to scripts/bench-cli-startup.ts, plus JSON output coverage and changed-target routing expectations for the new test-helper importer.
  • PR surface: Tests +77, Other +109. Total +186 across 4 files.
  • Reproducibility: not applicable. as a feature/tooling PR. The prior PR defects were source-proven in review comments and the current head addresses them; I did not run local tests because this review was read-only.

Automerge notes:

  • Ran the ClawSweeper repair loop before final review.
  • Included post-review commit in the final squash: test(perf): compare saved CLI startup benchmarks

The automerge loop is complete.

Automerge progress:

  • 2026-06-19 06:23:38 UTC review queued a5dbf6292fde (after repair)
  • 2026-06-19 08:33:21 UTC review requested repair a5dbf6292fde (structured ClawSweeper marker: fix-required (finding=review-feedback sha=a5dbf6...)
  • 2026-06-19 09:23:57 UTC review queued 1afa110f1bc8 (after repair)
  • 2026-06-19 09:37:19 UTC review passed 1afa110f1bc8 (structured ClawSweeper verdict: pass (sha=1afa110f1bc86beb77fd434b3edd2c75cfb33...)
  • 2026-06-19 09:37:50 UTC merged 1afa110f1bc8 (merged by ClawSweeper automerge)

@clawsweeper
clawsweeper Bot force-pushed the clawsweeper/automerge-openclaw-openclaw-85400 branch from 32da7b8 to b9f2269 Compare June 19, 2026 05:50
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jun 19, 2026
@clawsweeper
clawsweeper Bot force-pushed the clawsweeper/automerge-openclaw-openclaw-85400 branch from b9f2269 to 981b466 Compare June 19, 2026 06:09
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jun 19, 2026
@clawsweeper
clawsweeper Bot force-pushed the clawsweeper/automerge-openclaw-openclaw-85400 branch from 981b466 to a5dbf62 Compare June 19, 2026 06:23
@clawsweeper
clawsweeper Bot force-pushed the clawsweeper/automerge-openclaw-openclaw-85400 branch from a5dbf62 to 1afa110 Compare June 19, 2026 09:25
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Jun 19, 2026
@clawsweeper
clawsweeper Bot merged commit 2e0dfda into main Jun 19, 2026
21 of 23 checks passed
@clawsweeper
clawsweeper Bot deleted the clawsweeper/automerge-openclaw-openclaw-85400 branch June 19, 2026 09:37
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jun 20, 2026
Summary:
- Adds saved CLI startup benchmark report comparison flags to `scripts/bench-cli-startup.ts`, plus JSON output coverage and changed-target routing expectations for the new test-helper importer.
- PR surface: Tests +77, Other +109. Total +186 across 4 files.
- Reproducibility: not applicable. as a feature/tooling PR. The prior PR defects were source-proven in review comments and the current head addresses them; I did not run local tests because this review was read-only.

Automerge notes:
- Ran the ClawSweeper repair loop before final review.
- Included post-review commit in the final squash: test(perf): compare saved CLI startup benchmarks

Validation:
- ClawSweeper review passed for head 1afa110.
- Required merge gates passed before the squash merge.

Prepared head SHA: 1afa110
Review: openclaw#94812 (comment)

Co-authored-by: clawsweeper <274271284+clawsweeper[bot]@users.noreply.github.com>
Co-authored-by: Felix Isaac Lim <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

clawsweeper:automerge Maintainer opted this PR into bounded ClawSweeper-reviewed automerge clawsweeper Tracked by ClawSweeper automation clownfish:automerge Maintainer opted this Clownfish PR into bounded ClawSweeper-reviewed automerge P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. scripts Repository scripts size: M status: 🚀 automerge armed This PR is in ClawSweeper's automerge lane.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants