Skip to content

[Bug]: Deep-profile release gate treats instrumented RSS as baseline-eligible #102888

Description

@vincentkoc

Bug type

Behavior bug (incorrect release-gate state without a product failure)

Beta release blocker

No. This is release infrastructure, not plugin compatibility.

Summary

The OpenClaw Performance workflow treats profiler-inflated CPU/RSS thresholds as baseline-eligible release failures even when Kova explicitly marks the measurements as profiling-affected and baseline-ineligible.

Steps to reproduce

  1. Dispatch OpenClaw Performance for candidate dcee1da876e683277c0092e8efdaf019e4dcc303 with the release profile, deep_profile=true, and fail_on_regression=true.
  2. Observe the normal mock and live lanes pass.
  3. Observe the deep-profile lane fail only because peak RSS exceeds the 900 MB release threshold.
  4. Inspect the Kova report and confirm the failed record has:
    • profilingEnabled: true
    • profilingAffectsResourceMeasurements: true
    • profilingBaselineEligible: false
    • only CPU/RSS resource violations

Expected behavior

The trusted report adapter should tolerate a non-zero Kova verdict only when every blocking failure maps one-to-one to a profiled record whose violations are exclusively profiling-affected CPU/RSS metrics. Functional, latency, auth, harness, timeout, error, and baseline-regression failures must remain blocking.

Actual behavior

The adapter accepts only filtered PARTIAL reports, so Kova's DO_NOT_SHIP exit remains blocking even when the report explicitly says the only failed measurements are distorted by instrumentation and are not baseline-eligible.

OpenClaw version

2026.7.1-beta.3 candidate at dcee1da876e683277c0092e8efdaf019e4dcc303

Operating system

GitHub Actions Ubuntu 24.04

Install method

OpenClaw Performance release workflow

Model

Mock OpenAI-compatible provider for the deep-profile lane; the sibling live GPT-5.5 lane passed.

Provider / routing chain

OpenClaw Performance -> Kova -> OpenClaw CLI agent path -> mock OpenAI-compatible provider

Additional provider/model setup details

The deep-profile lane uses Kova profiling instrumentation. The corresponding normal mock-provider lane and live-provider lane passed on the same candidate.

Logs

Initial run: https://github.com/openclaw/openclaw/actions/runs/29022423555
Deep-profile retry: https://github.com/openclaw/openclaw/actions/runs/29023140517
Observed peak RSS: 925 MB, then 924 MB (threshold: 900 MB)
Kova source ref: 4f1460165830

Screenshots, recordings, and evidence

The retry artifact's release report records profilingAffectsResourceMeasurements=true, profilingBaselineEligible=false, and only peakRssMb / resourceByRole.*.peakRssMb violations.

Impact and severity

Affected: maintainers running instrumented release performance checks.

Severity: high operational impact during release closeout; no demonstrated product regression.

Frequency: reproduced in two consecutive deep-profile runs.

Consequence: a valid release candidate is blocked and the same expensive lane is rerun without changing product behavior.

Additional information

Pinned Kova source defines profiling as affecting resource measurements and makes profiled records baseline-ineligible. The OpenClaw adapter should enforce that contract narrowly and fail closed for every other gate card or violation kind.

Metadata

Metadata

Assignees

Labels

P2Normal backlog priority with limited blast radius.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:otherThis issue has meaningful maintainer-visible impact outside the owned taxonomy.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.maintainerMaintainer-authored PR

Type

No type

Fields

Priority

None yet

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions