test: save ~79 CI hours/mo in gateway session utils by zats · Pull Request #95602 · openclaw/openclaw

zats · 2026-06-21T17:58:23Z

What Problem This Solves

src/gateway/session-utils.test.ts had a gateway behavior test that accidentally loaded real bundled provider policy metadata to answer one fixture question: whether the per-agent DeepSeek model exposes xhigh thinking.

That made the gateway-core shard spend seconds in bundled provider/plugin infrastructure while the test's actual goal was narrower: verify listAgentsForGateway uses the agent-specific model when returning thinking controls.

CI impact estimate from the 30-day actions report:

Report window: 30 days, generated 2026-06-19.
CI/checks-node-agentic-gateway-core: estimated 36,902 executions/month, avg 99.3s, p50 92s, p95 151s.
Sample event mix for this job: 207/258 sampled executions were pull_request (80.2%).
Local paired full-file timing showed Vitest tests phase improved by 7.66s/run on average.
Estimated total CI runner savings: 36,902 * 7.66s = 78.5 runner-hours/month.
Estimated PR-facing subset: 78.5h * 80.2% = 63.0 runner-hours/month.

Why This Change Was Made

The test now installs a tiny in-test provider registry for MiniMax and DeepSeek, then asserts the DeepSeek thinking resolver is called with modelId: "deepseek-v4-flash".

This keeps the gateway behavior coverage focused:

per-agent model is used
returned agent model stays deepseek/deepseek-v4-flash
thinkingDefault: "xhigh" survives
DeepSeek thinking levels are used
thinkingOptions still mirrors thinkingLevels

Bundled DeepSeek provider-policy behavior remains covered by the DeepSeek plugin tests.

User Impact

No production behavior changes. This is a test-only CI speedup for the gateway-core shard, reducing feedback time and runner spend across CI, including PR runs.

Evidence

Focused proof after rebasing on latest origin/main:

OPENCLAW_VITEST_MAX_WORKERS=1 corepack pnpm exec vitest run --config test/vitest/vitest.gateway-core.config.ts src/gateway/session-utils.test.ts --reporter=verbose

Result:

Test Files  1 passed (1)
Tests       109 passed (109)
Target test 165ms
Duration    24.51s (tests 17.26s)

Before/after timing, 3 paired runs of the full src/gateway/session-utils.test.ts file, alternating revisions with a fresh Vitest fs-module cache path per run:

Wall clock avg:          32.54s -> 24.75s  (-7.79s, -24%)
Vitest total avg:        22.90s -> 14.57s  (-8.33s, -36%)
Vitest tests phase avg:  15.58s -> 7.91s   (-7.66s, -49%)
Target test avg:         7.90s  -> 0.11s   (-7.80s, -99%)

Other checks:

corepack pnpm exec oxfmt --check src/gateway/session-utils.test.ts
git diff --check
.agents/skills/autoreview/scripts/autoreview --mode branch --base origin/main

Autoreview result:

autoreview clean: no accepted/actionable findings reported

clawsweeper · 2026-06-21T18:00:06Z

Codex review: needs maintainer review before merge. Reviewed June 22, 2026, 4:11 AM ET / 08:11 UTC.

Summary
The PR changes src/gateway/session-utils.test.ts so one listAgentsForGateway thinking-default test uses an in-test MiniMax/DeepSeek provider registry instead of loading bundled provider policy metadata.

PR surface: Source -119, Tests +430, Other +24. Total +335 across 47 files.

Reproducibility: not applicable. this is a test-only CI hot-path cleanup. The PR body provides terminal timing proof for the changed gateway-core test path.

Review metrics: 1 noteworthy metric.

Benchmark proof: tests phase -7.66s/run; target test 7.90s -> 0.11s. This quantifies the claimed gateway-core CI hot-path improvement without relying on broad PR surface stats.

Stored data model
Persistent data-model change detected: database schema: extensions/qa-lab/src/test-file-scenario-runner.test.ts, database schema: src/config/sessions/store.pruning.test.ts, migration/backfill/repair: qa/scenarios/memory/active-memory-preprompt-recall.yaml, migration/backfill/repair: src/commands/doctor-config-flow.test.ts, migration/backfill/repair: src/commands/doctor/shared/preview-warnings.test.ts, migration/backfill/repair: src/commands/doctor/shared/preview-warnings.ts, and 13 more. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Risk before merge

[P1] A reviewer suggested a cache-based alternative and the author said they would yield, but no concrete replacement PR exists yet.
[P1] The PR's available potential merge commit predates latest current main, so maintainers should refresh/rebase and rerun relevant checks before landing.

Maintainer options:

Decide the mitigation before merge
Land the focused test-local provider fixture if maintainers accept it, or replace it with a concrete cache-based PR only after that alternative exists and is proven.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

[P2] The remaining action is maintainer judgment between landing this focused fixture now or waiting for a concrete cache-based replacement, plus refreshing the stale base before merge.

Security
Cleared: The intended merge surface is a colocated Vitest test change and adds no dependency, workflow, package, secret, or runtime execution surface.

Review details

Best possible solution:

Land the focused test-local provider fixture if maintainers accept it, or replace it with a concrete cache-based PR only after that alternative exists and is proven.

Do we have a high-confidence way to reproduce the issue?

Not applicable; this is a test-only CI hot-path cleanup. The PR body provides terminal timing proof for the changed gateway-core test path.

Is this the best way to solve the issue?

Yes for the stated goal: using the existing active-registry test seam is a narrow maintainable way to avoid bundled provider loading while preserving the per-agent model-selection invariant. The cache alternative is plausible but not yet a concrete superseding implementation.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 4e9dc6b5d5c4.

Label changes

Label justifications:

P3: This is a low-risk test-only CI performance cleanup with no intended production behavior change.
rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes terminal proof from the focused gateway-core Vitest command plus paired before/after timing for the changed test path.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes terminal proof from the focused gateway-core Vitest command plus paired before/after timing for the changed test path.

Evidence reviewed

PR surface:

Source -119, Tests +430, Other +24. Total +335 across 47 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	21	123	242	-119
Tests	20	592	162	+430
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	6	46	22	+24
Total	47	761	426	+335

What I checked:

Root and scoped policy read: Root AGENTS.md, src/gateway/AGENTS.md, extensions/AGENTS.md, ui/AGENTS.md, and scripts/AGENTS.md were read; the gateway guide specifically favors avoiding bundled plugin runtime in gateway tests when static descriptors are enough. (src/gateway/AGENTS.md:1, 4e9dc6b5d5c4)
Current main lacks the test-local registry fixture: On current main, the per-agent thinking-default test still calls listAgentsForGateway directly after constructing config, so the PR's registry isolation is not already implemented on main. (src/gateway/session-utils.test.ts:1693, 4e9dc6b5d5c4)
PR head adds focused provider fixture: PR head adds a local MiniMax/DeepSeek provider registry and asserts the DeepSeek resolver receives provider: deepseek with modelId: deepseek-v4-flash, matching the stated hot-path test goal. (src/gateway/session-utils.test.ts:1693, 20ab4716cebd)
Runtime path under test uses per-agent model: listAgentsForGateway resolves each agent's provider/model and passes that pair into thinking-level/default resolution, which is the invariant the PR locks down. (src/gateway/session-utils.ts:1299, 4e9dc6b5d5c4)
Bundled DeepSeek behavior remains covered elsewhere: DeepSeek plugin tests still cover V4 thinking levels and deepseek-v4-flash, so the gateway test does not need to load bundled DeepSeek policy to prove provider semantics. (extensions/deepseek/index.test.ts:309, 4e9dc6b5d5c4)
Related replacement search: Live GitHub PR searches found this PR and related broader gateway/cache work, but no open replacement PR for the specific session-utils bundled-provider-loading cleanup.

Likely related people:

jbetala7: Merged PR Fix Control UI agent thinking defaults #81795 introduced the agent thinking-default gateway behavior and touched the same gateway test/runtime surface. (role: introduced behavior; confidence: high; commits: f8f52592c595; files: src/gateway/session-utils.ts, src/gateway/session-utils.test.ts)
rolandrscheel: Merged PR Fix/sessions list resolver cache #77187 added session-utils resolver-cache performance work, which is the cache-oriented alternative discussed in this PR thread. (role: adjacent performance owner; confidence: medium; commits: e4cee2eb697f; files: src/gateway/session-utils.ts, src/gateway/session-utils.perf.test.ts)
Ayaan Zaidi: Current blame on the nearby src/gateway/session-utils.test.ts block points to a recent release/test commit, though the line history is shallow and less specific than the merged feature PR. (role: recent area contributor; confidence: low; commits: b3b8b289dd80; files: src/gateway/session-utils.test.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

xuwei-xy · 2026-06-21T18:01:24Z

This could be resolved by updating cache. I'll prepare a PR if that sounds good.

zats · 2026-06-21T18:02:17Z

sounds good, I'll close mine in favour of yours when ready

vincentkoc · 2026-06-22T10:03:17Z

Merged via squash.

Prepared head SHA: 53574bd4d1f770b8f143f2f2b443a6794c888375
Merge commit: 0c1f9635324bfba9e8d3897a769a62f3a119bb63

Thanks @zats!

@vincentkoc

Merged via squash. Prepared head SHA: 53574bd Co-authored-by: zats <[email protected]> Co-authored-by: vincentkoc <[email protected]> Reviewed-by: @vincentkoc

openclaw-barnacle Bot added gateway Gateway runtime size: XS labels Jun 21, 2026

zats changed the title ~~test: save ~63 PR CI hours/mo in gateway session utils~~ test: save ~79 CI hours/mo in gateway session utils Jun 21, 2026

totobusnello mentioned this pull request Jun 21, 2026

fix(agents): preserve streamed assistant text when Claude CLI result event is empty #90450

Merged

3 tasks

vincentkoc force-pushed the codex/session-utils-thinking-fixture branch 3 times, most recently from 0ac4785 to ccf42a8 Compare June 22, 2026 06:39

openclaw-barnacle Bot added size: S channel: slack Channel integration: slack app: web-ui App: web-ui cli CLI command changes commands Command implementations size: L and removed size: XS size: S labels Jun 22, 2026

vincentkoc self-assigned this Jun 22, 2026

openclaw-barnacle Bot added channel: msteams Channel integration: msteams scripts Repository scripts extensions: qa-lab extensions: llama-cpp size: XL and removed size: L labels Jun 22, 2026

vincentkoc force-pushed the codex/session-utils-thinking-fixture branch from 20ab471 to 26af4bd Compare June 22, 2026 08:35

vincentkoc force-pushed the codex/session-utils-thinking-fixture branch 5 times, most recently from 9bffece to e4ac00b Compare June 22, 2026 09:48

test: avoid bundled provider loading in session utils

53574bd

vincentkoc force-pushed the codex/session-utils-thinking-fixture branch from e4ac00b to 53574bd Compare June 22, 2026 09:52

vincentkoc merged commit 0c1f963 into openclaw:main Jun 22, 2026
91 checks passed

eldar702 mentioned this pull request Jun 22, 2026

fix(i18n): correct zh-CN theme/cron labels and translate Dreaming restart modal [AI-assisted] #94215

Open

clawsweeper Bot mentioned this pull request Jun 28, 2026

docs+tests: clarify agents.list visibility scope across CLI and Gateway #79985

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

test: save ~79 CI hours/mo in gateway session utils#95602

test: save ~79 CI hours/mo in gateway session utils#95602
vincentkoc merged 1 commit into
openclaw:mainfrom
zats:codex/session-utils-thinking-fixture

zats commented Jun 21, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jun 21, 2026 •

edited

Loading

Uh oh!

xuwei-xy commented Jun 21, 2026

Uh oh!

zats commented Jun 21, 2026

Uh oh!

Uh oh!

vincentkoc commented Jun 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

zats commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

Uh oh!

clawsweeper Bot commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xuwei-xy commented Jun 21, 2026

Uh oh!

zats commented Jun 21, 2026

Uh oh!

Uh oh!

vincentkoc commented Jun 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

zats commented Jun 21, 2026 •

edited

Loading

clawsweeper Bot commented Jun 21, 2026 •

edited

Loading