Skip to content

fix(ci): shard scoped e2e runs to avoid snapshot timeouts - #8236

Merged
pbrolin47 merged 2 commits into
mermaid-js:developfrom
Raj4478:bug/8235_shard-scoped-e2e
Sep 15, 2026
Merged

pbrolin47 merged 2 commits into
mermaid-js:developfrom
Raj4478:bug/8235_shard-scoped-e2e

Conversation

@Raj4478

@Raj4478 Raj4478 commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

📑 Summary

Scoped diagram changes still include the global Markdown snapshot runner. With a single CI shard, that work is close to the 30-minute job limit; #8235 documents three timed-out runs on unrelated contributions.

Run scoped E2E jobs on two shards and update the nearby comment to explain why. This distributes the snapshot workload using the workflow's existing sharding support.

Resolves #8235

📏 Design Decisions

  • Use two scoped shards to reduce the per-runner test workload while keeping the scoped matrix smaller than the eight-shard full suite.
  • The Playwright command already derives its denominator from strategy.job-total, and fullyParallel: true allows the single snapshot spec to be split between workers.
  • Screenshot and coverage artifacts already have shard-specific names. Argos downloads screenshots-* with merge-multiple: true and waits for the complete E2E matrix to succeed.
  • Keep the complete selected test set and the existing 30-minute limit.

Validation

  • Executed the workflow's actual scope-detection Bash and scripts/e2e-diagram-scope.mjs in temporary Git repositories. Before the change, five scoped entry paths selected one shard; afterwards, all 11 checked scenarios passed: diagram changes, fixture-only changes, diagram plus docs, merge queue, manual scoped/full runs, shared-renderer changes, workflow changes, the kill switch, direct develop pushes, and docs-only skipping.
  • Used Playwright 1.62.1 in --list mode on a registration-only suite built from the current 1,228 Markdown fixture paths, preserving the snapshot runner's nested suites and parallel setting. The two shards contain 614 + 614 tests, with zero overlap and zero omissions. This verifies scheduling, not diagram rendering.
  • Checked YAML parsing, the dynamic shard total, artifact naming/collection, and the success gate. git diff --check passed.
  • Not run locally: the full Mermaid build and visual E2E suite. Dependency downloads are restricted in this environment. CI and a scoped run should confirm actual wall-clock improvement before merging.

📋 Tasks

  • Read the contribution guidelines.
  • Validated the changed CI workflow and shard partitioning.
  • Updated the inline workflow documentation.
  • No package changeset needed: CI configuration only.

Summary

  • Updated scoped E2E runs to use two shards instead of one.
  • Distributed the global mmd-snapshots workload while retaining the full selected test set and 30-minute job limit.
  • Documented the sharding rationale in .github/workflows/e2e.yml.
  • Preserved existing sharding, artifact naming, Argos collection, and success-gate behavior.

Validation

  • Verified scope detection, shard partitioning, YAML parsing, artifact handling, and configuration.
  • Full build and visual E2E execution were not run locally due to dependency restrictions.

@changeset-bot

changeset-bot Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: b7c1c6c

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@netlify

netlify Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for mermaid-js ready!

Name Link
🔨 Latest commit b7c1c6c
🔍 Latest deploy log https://app.netlify.com/projects/mermaid-js/deploys/6aa803c622a6560008d62ba3
😎 Deploy Preview https://deploy-preview-8236--mermaid-js.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions github-actions Bot added the Type: Bug / Error Something isn't working or is incorrect label Sep 9, 2026
@coderabbitai

coderabbitai Bot commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 3ad19afa-fb6c-423d-9c08-3489e8476df9

📥 Commits

Reviewing files that changed from the base of the PR and between fe0e237 and becc278.

📒 Files selected for processing (1)
  • .github/workflows/e2e.yml

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.


📝 Walkthrough

Walkthrough

Scoped E2E runs now execute across two containers instead of one. The workflow comment documents that scoped runs include the global mmd snapshots.

Changes

Scoped E2E execution

Layer / File(s) Summary
Shard scoped E2E runs
.github/workflows/e2e.yml
E2E_SCOPED_MATRIX now uses shards [1,2] instead of [1]. The comment explains why scoped runs remain sharded.

Priority: ➖ Normal

Estimated code review effort: 1 (Trivial) | ~3 minutes

Severity of issue fixed: Medium

Suggested reviewers: sidharthv96, aloisklink, claude

Merge Risk: ⚪ Minimal · up to b7c1c

Scoped E2E runs will distribute the global Markdown snapshot workload across two shards, reducing timeout risk while retaining the existing test coverage and workflow behavior. No current merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the main change: sharding scoped E2E runs to prevent snapshot timeouts.
Description check ✅ Passed The description follows the repository template. It includes the summary, linked issue, design decisions, validation details, and completed task checklist.
Linked Issues check ✅ Passed The workflow changes satisfy issue #8235 by changing scoped E2E runs from one shard to two shards while preserving the complete snapshot workload, timeout limit, artifact handling, and success gate.
Out of Scope Changes check ✅ Passed The changes are limited to the scoped E2E shard matrix and its explanatory comment. Both changes directly support issue #8235 and the pull request objectives.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@pkg-pr-new

pkg-pr-new Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

@mermaid-js/examples

npm i https://pkg.pr.new/@mermaid-js/examples@8236

mermaid

npm i https://pkg.pr.new/mermaid@8236

@mermaid-js/layout-elk

npm i https://pkg.pr.new/@mermaid-js/layout-elk@8236

@mermaid-js/layout-tidy-tree

npm i https://pkg.pr.new/@mermaid-js/layout-tidy-tree@8236

@mermaid-js/mermaid-zenuml

npm i https://pkg.pr.new/@mermaid-js/mermaid-zenuml@8236

@mermaid-js/parser

npm i https://pkg.pr.new/@mermaid-js/parser@8236

@mermaid-js/tiny

npm i https://pkg.pr.new/@mermaid-js/tiny@8236

commit: b7c1c6c

@codecov

codecov Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.73%. Comparing base (c4251fe) to head (b7c1c6c).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           develop    #8236     +/-   ##
==========================================
  Coverage    80.73%   80.73%             
==========================================
  Files          619      619             
  Lines        84970    84970             
  Branches     16091    18596   +2505     
==========================================
  Hits         68601    68601             
  Misses       15350    15350             
  Partials      1019     1019             
Flag Coverage Δ
e2e 73.63% <ø> (ø)
unit 78.80% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@argos-ci

argos-ci Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Argos notifications ↗︎

Build Status Details Updated (UTC)
default (Inspect) 👍 Approved by pbrolin47 1 changed Sep 14, 2026, 2:48 PM

@pbrolin47

Copy link
Copy Markdown
Collaborator

Hi @Raj4478,
Thanks for this PR and thanks @filipsajdak for identifying/analysing the issue.

[sisyphos-bot]
What's Working Well

🎉 [praise] The rebase made this a clean, minimal fix. The 4-line change (E2E_SCOPED_MATRIX: '[1]' → [1,2] plus an updated comment) now correctly targets the infrastructure that's in develop: the
detect-scope script uses set_output matrix "$E2E_SCOPED_MATRIX" for scoped runs, so this single value change propagates to every scoped-run scenario.

🎉 [praise] The sharding mechanics are sound end-to-end:

  • Playwright's --shard="${SHARD}/${SHARD_TOTAL}" (where SHARD_TOTAL = strategy.job-total = 2) distributes tests with zero overlap and zero omissions
  • Screenshot artifacts screenshots-1 / screenshots-2 are collected by argos-batch via pattern: screenshots-* + merge-multiple: true
  • argos-batch gates on needs.e2e.result == 'success', which requires all matrix shards to pass — no partial Argos uploads possible
  • Playwright names screenshots by spec-relative path, so there's no shard-dependent renaming needed (unlike the old cypress-split setup)

🎉 [praise] The PR description's validation is thorough and matches the actual workflow — 11 scoping scenarios, 614+614 shard partitioning, artifact naming. The author clearly understands the
workflow deeply.


Security

No security issues. This is a CI env var value change only. All run: steps use $VARIABLE (bash), not ${{ inputs.* }} interpolation, so there's no script injection surface. Actions are SHA-pinned
throughout.


Self-Check

  • At least one 🎉 [praise] ✓
  • No 🔴 or 🟡 issues ✓
  • Severity tally: 0 🔴 / 0 🟡 / 0 🟢 / 0 💡 / 3 🎉
  • Verdict: APPROVE ✓
  • Not a draft ✓
  • Tone: appreciative ✓

@pbrolin47
pbrolin47 added this pull request to the merge queue Sep 15, 2026
Merged via the queue into mermaid-js:develop with commit cf83441 Sep 15, 2026
31 checks passed
@mermaid-bot

mermaid-bot Bot commented Sep 15, 2026

Copy link
Copy Markdown

@Raj4478, Thank you for the contribution!
You are now eligible for a year of Premium account on MermaidChart.
Sign up with your GitHub account to activate.

mir-ashiq pushed a commit to mir-ashiq/mermaid that referenced this pull request Sep 16, 2026


Develop picked up mermaid-js#8236 (shard scoped e2e runs across two containers),
which fixes the 30-minute timeout that has been cancelling the scoped
e2e job on diagram-scoped PRs like this one. No source changes — only
.github/workflows/e2e.yml — so the parser and specs are untouched from
the previously verified state (lint:jison clean, sequence suite 148/148).
akashchamp pushed a commit to akashchamp/mermaid that referenced this pull request Sep 25, 2026
…-e2e

fix(ci): shard scoped e2e runs to avoid snapshot timeouts
akashchamp pushed a commit to akashchamp/mermaid that referenced this pull request Sep 25, 2026


Develop picked up mermaid-js#8236 (shard scoped e2e runs across two containers),
which fixes the 30-minute timeout that has been cancelling the scoped
e2e job on diagram-scoped PRs like this one. No source changes — only
.github/workflows/e2e.yml — so the parser and specs are untouched from
the previously verified state (lint:jison clean, sequence suite 148/148).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Type: Bug / Error Something isn't working or is incorrect

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI: scoped e2e runs the full mmd-snapshots suite on one shard and now exceeds the 30-minute cap

2 participants