Skip to content

[Bug]: Release performance CI succeeds when report publishing fails #103125

Description

@steipete

Bug type

Behavior bug (incorrect output/state without crash)

Beta release blocker

No

Summary

Release-profile performance CI can report success even when its required Clawgrit baseline report was never published.

Steps to reproduce

  1. Run the release-profile OpenClaw Performance workflow with fail_on_regression=true.
  2. Let Kova finish successfully while the Clawgrit reports push fails.
  3. Observe that the workflow retries the push, emits only a warning, and still concludes successfully.

Observed in run https://github.com/openclaw/openclaw/actions/runs/29051458125: the report push failed authentication five times, then the job exited successfully with an optional-publish warning.

Expected behavior

Scheduled and release/full performance gates must fail when a required baseline report cannot be published. A manually dispatched non-release run may remain advisory because it is not authoritative release evidence.

Actual behavior

Run 29051458125 completed successfully despite Invalid username or token, five failed pushes, and no published report for the tested SHA.

OpenClaw version

4da2ee3514fa50b7f0233053162f5f0f3f39896c

Operating system

Ubuntu 24.04 GitHub Actions runner

Install method

Repository checkout in GitHub Actions

Model

Mock provider performance lane

Provider / routing chain

OpenClaw performance workflow -> Kova mock-provider evaluation -> Clawgrit reports repository

Additional provider/model setup details

profile=release, repeat=3, and fail_on_regression=true.

Logs

remote: Invalid username or token. Password authentication is not supported for Git operations.
fatal: Authentication failed for 'https://github.com/openclaw/clawgrit-reports.git/'
Kova artifacts uploaded, but optional clawgrit report publish failed after 5 attempts.

Screenshots, recordings, and evidence

https://github.com/openclaw/openclaw/actions/runs/29051458125/job/86232944434

Impact and severity

  • Affected: release maintainers and every release/full or scheduled performance gate that depends on durable historical baselines.
  • Severity: High for release preparation because a green workflow can omit authoritative regression evidence.
  • Frequency: Every required run whose report push fails under the current advisory-only publisher.
  • Consequence: release CI can be green without advancing the baseline repository or latest-lane pointer.

Additional information

The publisher should run on a fresh artifact-consuming runner, keep the target checkout away from the report credential, validate report paths, serialize/retry concurrent writers, and expose the report URL only after a successful push.

Metadata

Metadata

Assignees

Labels

P2Normal backlog priority with limited blast radius.bugSomething isn't workingclawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:otherThis issue has meaningful maintainer-visible impact outside the owned taxonomy.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.maintainerMaintainer-authored PR

Type

No type

Fields

Priority

None yet

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions