🔧 Add spec_filter input to Playwright workflow#10782
Conversation
Allows running individual spec files via workflow_dispatch instead of the full 4-shard test suite, enabling faster debugging of CI-only test failures. Signed-off-by: Andrew Anderson <[email protected]>
kubestellar-prow
Bot
added
the
dco-signoff: yes
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
✅ Deploy Preview for kubestellarconsole ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
👋 Hey @clubanderson — thanks for opening this PR!
This is an automated message. |
kubestellar-prow
Bot
added
the
size/S
There was a problem hiding this comment.
Pull request overview
Adds a spec_filter input to the Playwright E2E workflow so a manually-triggered run can target a single spec file instead of the full sharded suite.
Changes:
- Add
workflow_dispatch.inputs.spec_filterto accept an optional spec path. - Update the Playwright test step to run either a single spec (when provided) or the existing 4-shard run (when empty).
| run: | | ||
| if [ -n "${{ inputs.spec_filter }}" ]; then | ||
| npx playwright test --project=chromium "${{ inputs.spec_filter }}" | ||
| else | ||
| npx playwright test --project=chromium --shard=${{ matrix.shard }}/4 | ||
| fi |
There was a problem hiding this comment.
With spec_filter set, this step still runs in every matrix shard (4 jobs). That means the same spec file will execute 4 times and produce 4 blob reports, which can also distort the merged report. To match the stated behavior (“only that file runs — no sharding”), gate the job/strategy so only a single shard runs when spec_filter is non-empty (e.g., job-level if for matrix.shard == 1, or a dynamic matrix / separate non-matrix job).
| if [ -n "${{ inputs.spec_filter }}" ]; then | ||
| npx playwright test --project=chromium "${{ inputs.spec_filter }}" | ||
| else | ||
| npx playwright test --project=chromium --shard=${{ matrix.shard }}/4 | ||
| fi | ||
| env: | ||
| PLAYWRIGHT_BASE_URL: http://localhost:4173 |
There was a problem hiding this comment.
inputs.spec_filter is interpolated directly into a bash script/command. For workflow_dispatch inputs this enables shell injection via strings like $(...) or backticks, since the value is inserted into the script before the shell runs. Safer pattern: pass the input via env (e.g. SPEC_FILTER: ${{ inputs.spec_filter }}) and reference it as "$SPEC_FILTER", optionally validating it matches an expected path pattern (like ^e2e/.*\.spec\.(ts|js)$) before executing Playwright.
| if [ -n "${{ inputs.spec_filter }}" ]; then | |
| npx playwright test --project=chromium "${{ inputs.spec_filter }}" | |
| else | |
| npx playwright test --project=chromium --shard=${{ matrix.shard }}/4 | |
| fi | |
| env: | |
| PLAYWRIGHT_BASE_URL: http://localhost:4173 | |
| if [ -n "$SPEC_FILTER" ]; then | |
| if [[ ! "$SPEC_FILTER" =~ ^e2e/.*\.spec\.(ts|js)$ ]]; then | |
| echo "Invalid spec_filter: must match e2e/*.spec.ts or e2e/*.spec.js" | |
| exit 1 | |
| fi | |
| npx playwright test --project=chromium "$SPEC_FILTER" | |
| else | |
| npx playwright test --project=chromium --shard=${{ matrix.shard }}/4 | |
| fi | |
| env: | |
| PLAYWRIGHT_BASE_URL: http://localhost:4173 | |
| SPEC_FILTER: ${{ inputs.spec_filter }} |
|
Thank you for your contribution! Your PR has been merged. Check out what's new:
Stay connected: Slack #kubestellar-dev | Multi-Cluster Survey |
|
Post-merge build verification passed ✅ Both Go and frontend builds compiled successfully against merge commit |
- Issue #1 (FIXED): Cluster name strict-mode violation — added .first() to lines 91-93 - Issue #2 (INVESTIGATING): Filter tabs not rendering in webkit/firefox tests - Issue #3 (INVESTIGATING): Multiple dashboard-page testid elements cause strict-mode - Triggered targeted Clusters.spec.ts run to validate fix - PR #10782 enabled spec_filter workflow capability Co-authored-by: Copilot <[email protected]> Signed-off-by: Copilot <[email protected]>
Allows running individual spec files via workflow_dispatch instead of the full 4-shard test suite, enabling faster debugging of CI-only test failures. Signed-off-by: Andrew Anderson <[email protected]> Signed-off-by: lightyagami2109 <[email protected]>
- Issue kubestellar#1 (FIXED): Cluster name strict-mode violation — added .first() to lines 91-93 - Issue kubestellar#2 (INVESTIGATING): Filter tabs not rendering in webkit/firefox tests - Issue kubestellar#3 (INVESTIGATING): Multiple dashboard-page testid elements cause strict-mode - Triggered targeted Clusters.spec.ts run to validate fix - PR kubestellar#10782 enabled spec_filter workflow capability Co-authored-by: Copilot <[email protected]> Signed-off-by: Copilot <[email protected]> Signed-off-by: lightyagami2109 <[email protected]>
Summary
spec_filterinput to the Playwright E2E workflow'sworkflow_dispatchtriggere2e/Sidebar.spec.ts), only that file runs — no shardingTest plan
spec_filter: e2e/Sidebar.spec.ts— verify only sidebar tests runspec_filter— verify full suite runs with shards🤖 Generated with Claude Code