Skip to content

Improve isolated test latency and inventory reliability #100018

Description

@steipete

Summary

Keep every individual test in the isolated full-suite inventory below one second without weakening behavioral coverage.

Problem to solve

The full isolated Vitest inventory contains tests whose wall time is dominated by repeated module initialization, oversized fixtures, real timeout waits, or broad plugin discovery. These outliers make local and CI feedback slower, obscure genuine regressions, and can push a monolithic profiling run into OOM before it reports the complete inventory.

Proposed solution

Add a bounded full-suite duration-report path, then remove the measured outliers at their owning boundaries: reuse shared harness warmup, defer heavy imports, keep plugin artifacts lightweight, add core-registry fast paths, reduce fixtures while preserving size-limit contracts, and replace real deadline sleeps with deterministic timer coverage. The final isolated inventory should report zero individual tests above 1,000 ms.

Alternatives considered

  • Raising the threshold hides the regressions and preserves the slow feedback loop.
  • Running only a hand-picked offender list misses new outliers elsewhere in the 89-plan suite.
  • Increasing worker counts makes timing noisier and reintroduces the memory-pressure failure mode.

Impact

Affected: OpenClaw contributors and CI runners.

Severity: Medium developer-productivity and CI-capacity cost.

Frequency: Every broad test or duration-profile run.

Consequence: Slower iteration, harder performance triage, and OOM-prone inventory runs.

Evidence/examples

The isolated inventory previously exposed repeated one-time initialization costs and real timer waits above one second. A bounded Testbox run avoids the monolithic OOM and produces per-plan RSS and individual-test timing evidence.

Additional information

This is maintainer-requested test/runtime performance work. It does not change release versions or published artifacts.

Metadata

Metadata

Assignees

Labels

P2Normal backlog priority with limited blast radius.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.clawsweeper:needs-infoClawSweeper needs more reporter information before it can verify this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.enhancementNew feature or requestimpact:otherThis issue has meaningful maintainer-visible impact outside the owned taxonomy.issue-rating: 🦪 silver shellfishThin issue quality; more reproduction proof or environment detail is needed.maintainerMaintainer-authored PR

Type

No type

Fields

Priority

None yet

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions