Skip to content

ci: generalize QA profile evidence workflow#95880

Merged
RomneyDa merged 2 commits into
mainfrom
ci/qa-profile-evidence-workflow
Jun 23, 2026
Merged

ci: generalize QA profile evidence workflow#95880
RomneyDa merged 2 commits into
mainfrom
ci/qa-profile-evidence-workflow

Conversation

@RomneyDa

@RomneyDa RomneyDa commented Jun 22, 2026

Copy link
Copy Markdown
Member

Summary

  • rename the manual/callable workflow to qa-profile-evidence.yml / QA Profile Evidence
  • add a required qa_profile input so callers can generate evidence for any taxonomy-backed QA profile, with manual dispatch still defaulting to release
  • rename the artifact and manifest to generic qa-profile-evidence-* names and expose qa_profile, artifact_name, target_sha, trusted_reason, and qa_evidence_path for reusable workflow consumers
  • expose qa_exit_code and qa_passed so consumers can use uploaded evidence even when the QA profile records failed scenarios
  • keep manual dispatch as a gate: manual runs still fail when the QA profile command exits non-zero; reusable callers can opt into that with fail_on_qa_failure: true
  • validate the requested profile against the parsed QA scorecard taxonomy before running the private QA runtime

Verification

  • go run github.com/rhysd/actionlint/cmd/[email protected] .github/workflows/qa-profile-evidence.yml
  • git diff --check origin/main...HEAD
  • .agents/skills/autoreview/scripts/autoreview --mode branch --base origin/main --parallel-tests "go run github.com/rhysd/actionlint/cmd/[email protected] .github/workflows/qa-profile-evidence.yml && git diff --check origin/main...HEAD"

Run follow-up

  • Watched https://github.com/openclaw/openclaw/actions/runs/27989312235 after the previous workflow merged.
  • The workflow infrastructure succeeded through ref validation, private QA runtime build, evidence validation, and artifact upload.
  • The uploaded artifact is release-qa-profile-evidence-89de454f822cac98505651fe3ee8340a054f3e37 with artifact ID 7807415546.
  • The run failed only after upload because the release QA profile returned exit code 1: 94 scenarios passed and 13 failed.
  • This PR keeps that manual gate behavior, but makes reusable workflow consumers receive qa_exit_code / qa_passed outputs and a valid uploaded evidence artifact by default.

@openclaw-barnacle openclaw-barnacle Bot added size: S maintainer Maintainer-authored PR labels Jun 22, 2026
@RomneyDa
RomneyDa merged commit fee8ab4 into main Jun 23, 2026
81 of 83 checks passed
@RomneyDa
RomneyDa deleted the ci/qa-profile-evidence-workflow branch June 23, 2026 00:33
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jun 23, 2026
* ci: generalize qa profile evidence workflow

* ci: keep qa evidence workflow usable on qa failures
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintainer Maintainer-authored PR size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant