Split rspec-package-tests into parallel generator/unit shards#3134
Split rspec-package-tests into parallel generator/unit shards#3134
Conversation
The RSpec package test job ran all ~1,013 examples in a single process (~24 min). Split it into two parallel matrix shards: - generators: spec/react_on_rails/generators (429 examples, heavy I/O) - unit: everything else under spec/react_on_rails (584 examples) This applies to both PR (latest-only) and main (latest + minimum) matrix configurations, cutting CI time roughly in half. Closes #3131 Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
WalkthroughThe CI test matrix gains a Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Greptile SummaryThis PR splits the Confidence Score: 5/5Safe to merge — changes are isolated to CI configuration with no runtime code impact. The only finding is a P2 style/consistency note about inline expressions in the matrix-setup step, which poses no real security risk given the values are trusted GitHub context outputs. All logic (shard routing, exclusion pattern, artifact naming, matrix dimensions) looks correct. No files require special attention. Important Files Changed
Sequence DiagramsequenceDiagram
participant GH as GitHub Actions
participant DC as detect-changes
participant SM as setup-gem-tests-matrix
participant G as rspec-package-tests (shard: generators)
participant U as rspec-package-tests (shard: unit)
GH->>DC: trigger (push / PR / dispatch)
DC-->>SM: docs_only, run_ruby_tests, has_full_ci_label
alt main branch OR full-ci label
SM-->>G: ruby 3.4 latest + ruby 3.2 minimum
SM-->>U: ruby 3.4 latest + ruby 3.2 minimum
else PR (latest only)
SM-->>G: ruby 3.4 latest
SM-->>U: ruby 3.4 latest
end
par Generators shard
G->>G: bundle exec rspec spec/react_on_rails/generators
G-->>GH: upload artifact (…-generators)
and Unit shard
U->>U: bundle exec rspec spec/react_on_rails --exclude-pattern **/generators/**
U-->>GH: upload artifact (…-unit)
end
Reviews (2): Last reviewed commit: "fix: use env var for matrix.shard to fol..." | Re-trigger Greptile |
Review: Split rspec-package-tests into parallel shardsOverall this is a clean, well-scoped CI-only change. The shard boundary (generators vs. everything else) is a sensible split — generators are the heaviest filesystem consumers — and the artifact naming update correctly prevents collisions. What looks good
Two items to address
One implicit behaviour to be aware of If elif [ "$SHARD" = "unit" ]; then
bundle exec rspec spec/react_on_rails --exclude-pattern "**/generators/**"
else
echo "Unknown shard: $SHARD" && exit 1
fiNot required for this PR, but worth considering if/when more shards are added. |
…ardening
Route ${{ matrix.shard }} through an env var instead of direct shell
interpolation, per GitHub's security hardening guide. Also switch from
POSIX [ ] to bash [[ ]] to match surrounding workflow style.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Review comments addressedI (Claude Code) triaged all 6 review comments and addressed them as follows: Fixed (1):
Acknowledged with rationale (2):
Skipped (3):
All 3 review threads have been resolved. |
|
Good optimization overall. Isolating heavy filesystem-I/O generator specs into their own shard is the right approach. No blockers, but a few notes: Exclude pattern edge case: The pattern Bash comparison style (minor): Long single-line JSON (readability): The matrix JSON on lines 91 and 94 are very long one-liners. Consider a multi-line variable or heredoc approach before echoing to Git Stuff step comment is misleading: The step runs for both shards when |
| # Full matrix: test both latest and minimum supported versions | ||
| echo 'matrix={"include":[{"ruby-version":"3.4","dependency-level":"latest"},{"ruby-version":"3.2","dependency-level":"minimum"}]}' >> $GITHUB_OUTPUT | ||
| # Full matrix: test both latest and minimum supported versions × 2 shards | ||
| echo 'matrix={"include":[{"ruby-version":"3.4","dependency-level":"latest","shard":"generators"},{"ruby-version":"3.4","dependency-level":"latest","shard":"unit"},{"ruby-version":"3.2","dependency-level":"minimum","shard":"generators"},{"ruby-version":"3.2","dependency-level":"minimum","shard":"unit"}]}' >> $GITHUB_OUTPUT |
There was a problem hiding this comment.
These long JSON one-liners are hard to read and extend. Consider a multi-line approach for maintainability:
| echo 'matrix={"include":[{"ruby-version":"3.4","dependency-level":"latest","shard":"generators"},{"ruby-version":"3.4","dependency-level":"latest","shard":"unit"},{"ruby-version":"3.2","dependency-level":"minimum","shard":"generators"},{"ruby-version":"3.2","dependency-level":"minimum","shard":"unit"}]}' >> $GITHUB_OUTPUT | |
| matrix=$(cat <<'EOF' | |
| {"include":[ | |
| {"ruby-version":"3.4","dependency-level":"latest","shard":"generators"}, | |
| {"ruby-version":"3.4","dependency-level":"latest","shard":"unit"}, | |
| {"ruby-version":"3.2","dependency-level":"minimum","shard":"generators"}, | |
| {"ruby-version":"3.2","dependency-level":"minimum","shard":"unit"} | |
| ]} | |
| EOF | |
| ) | |
| echo "matrix=$matrix" >> $GITHUB_OUTPUT |
This makes adding or removing matrix entries much easier to review in diffs.
| SHARD: ${{ matrix.shard }} | ||
| run: | | ||
| cd react_on_rails | ||
| if [[ "$SHARD" = "generators" ]]; then |
There was a problem hiding this comment.
Style nit: prefer == over = for string comparison in [[ ]] — both are valid in bash but == is the conventional form and more clearly signals "comparison" vs "assignment".
| if [[ "$SHARD" = "generators" ]]; then | |
| if [[ "$SHARD" == "generators" ]]; then |
| # Prevent Minitest's at_exit runner from parsing RSpec-only CLI args after | ||
| # rails/test_help loads ActiveSupport's autorun hooks. | ||
| # rubocop:disable Style/ClassVars | ||
| Minitest.class_variable_set(:@@installed_at_exit, true) if Minitest.class_variable_defined?(:@@installed_at_exit) |
There was a problem hiding this comment.
This patches a private Minitest implementation detail (@@installed_at_exit). It works today, but it could silently break if Minitest renames or removes that class variable in a future version — with no error, just the autorun hook re-appearing.
A slightly more resilient alternative is to prevent the hook from firing without touching internals:
| Minitest.class_variable_set(:@@installed_at_exit, true) if Minitest.class_variable_defined?(:@@installed_at_exit) | |
| Minitest.class_variable_set(:@@installed_at_exit, true) if Minitest.class_variable_defined?(:@@installed_at_exit) |
The current guard (class_variable_defined?) already mitigates the worst-case (no-op on unknown Minitest versions), so this is low risk. Just worth noting in a follow-up that the real fix would be avoiding require "rails/test_help" entirely if it's not needed, since that's the source of the autorun registration.
| # Full matrix: test both latest and minimum supported versions | ||
| echo 'matrix={"include":[{"ruby-version":"3.4","dependency-level":"latest"},{"ruby-version":"3.2","dependency-level":"minimum"}]}' >> $GITHUB_OUTPUT | ||
| # Full matrix: test both latest and minimum supported versions × 2 shards | ||
| echo 'matrix={"include":[{"ruby-version":"3.4","dependency-level":"latest","shard":"generators"},{"ruby-version":"3.4","dependency-level":"latest","shard":"unit"},{"ruby-version":"3.2","dependency-level":"minimum","shard":"generators"},{"ruby-version":"3.2","dependency-level":"minimum","shard":"unit"}]}' >> $GITHUB_OUTPUT |
There was a problem hiding this comment.
These long one-liner JSON strings are hard to read and diff. GitHub Actions supports multi-line echo with >>, so you can split this cleanly:
| echo 'matrix={"include":[{"ruby-version":"3.4","dependency-level":"latest","shard":"generators"},{"ruby-version":"3.4","dependency-level":"latest","shard":"unit"},{"ruby-version":"3.2","dependency-level":"minimum","shard":"generators"},{"ruby-version":"3.2","dependency-level":"minimum","shard":"unit"}]}' >> $GITHUB_OUTPUT | |
| echo 'matrix={"include":[' \ | |
| '{"ruby-version":"3.4","dependency-level":"latest","shard":"generators"},' \ | |
| '{"ruby-version":"3.4","dependency-level":"latest","shard":"unit"},' \ | |
| '{"ruby-version":"3.2","dependency-level":"minimum","shard":"generators"},' \ | |
| '{"ruby-version":"3.2","dependency-level":"minimum","shard":"unit"}' \ | |
| ']}' >> $GITHUB_OUTPUT |
Not a correctness issue — just a maintainability nit for when the matrix inevitably grows again.
| if [[ "$SHARD" = "generators" ]]; then | ||
| bundle exec rspec spec/react_on_rails/generators | ||
| else | ||
| bundle exec rspec spec/react_on_rails --exclude-pattern "**/generators/**" |
There was a problem hiding this comment.
Worth confirming the --exclude-pattern glob behaves as expected with RSpec. The pattern **/generators/** excludes files inside a generators/ subdirectory, but spec files directly named generators_spec.rb at the top level (if any were ever added) would still be picked up by the unit shard. Currently the directory structure only has spec/react_on_rails/generators/*.rb, so this is fine — just something to keep in mind if flat spec files named after generators are added later.
Review: Split rspec-package-tests into parallel generator/unit shardsOverall: Clean, well-motivated change. The shard split correctly targets the I/O-heavy generator specs and the artifact naming is updated consistently. A few observations below. Correctness
Minitest suppression (both spec_helpers)The new One question: Artifact path (pre-existing)The No blocking issues. The three inline comments are nits/notes — the implementation is sound and achieves the stated ~12 min reduction goal. |
…ages * origin/main: Fix initial page startup race for late-loading client bundles (#3151) chore: apply prettier formatting to tracked docs files (#3153) docs: comprehensive RSC API documentation and registration consolidation (#3140) Split rspec-package-tests into parallel generator/unit shards (#3134) fix: add concurrency groups to long-running CI workflows (#3133) refactor: add RenderRequest, JsCodeBuilder, and RenderingStrategy abstractions (#3094) fix: address deferred review items from PR #2849 (#3093) Add complimentary OSS license policy for React on Rails Pro (#3123) fix: centralize CI docs-only detection and add CLI flag validation (#3091) refactor: replace stub-throw + Object.assign with capability-based composition (#3096) Enhance address-review with parallel fixes, self-review, and Greptile verification (#3121) fix: Doctor no longer fails custom projects for missing bin/dev (#3117) fix: cap webpack <5.106.0 to prevent ExecJS SSR breakage (#3095) Add Rspack + RSC compatibility tests and documentation (#1828) (#3120) Add error scenarios hub and test pages (#2497) docs: document polyfill requirements for web-targeted server bundles (#3092) docs: RSC integration pitfalls from tutorial app (#3087) docs: fix render function/helper API documentation (#3088) Doctor: accept TS/TSX server bundle suffixes (#3111) feat: add CI guard requiring sidebar updates when adding docs (#3089)
## Summary - Splits the `rspec-package-tests` CI job into two parallel matrix shards: **generators** (429 examples with heavy filesystem I/O) and **unit** (584 examples covering everything else) - Applies to both PR (latest-only) and main (latest + minimum) matrix configurations - Artifact names updated to include shard name to avoid collisions Expected CI time reduction: ~24 min → ~12 min. Closes #3131 ## Test plan - [ ] Verify both `generators` and `unit` shard jobs appear in the CI matrix - [ ] Confirm generator shard runs only `spec/react_on_rails/generators` - [ ] Confirm unit shard runs `spec/react_on_rails` excluding generators - [ ] Check artifact names are unique per shard - [ ] Verify full matrix on main includes 4 jobs (2 Ruby versions × 2 shards) 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Medium risk because it changes CI execution by splitting the RSpec suite into shard-specific commands and adjusts test initialization to suppress Minitest/Test::Unit autorun behavior, which could affect what tests run and how failures surface. > > **Overview** > Splits `rspec-package-tests` into two parallel matrix shards (`generators` vs `unit`) for both PR and full (main/`full-ci`) runs, and runs shard-specific RSpec commands (generators-only vs everything excluding generators). > > Updates uploaded artifact names to include the shard to avoid collisions, and tweaks both `spec_helper.rb` files to `require "minitest"` and disable Minitest/Test::Unit autorun hooks so RSpec CLI args don’t get parsed by Minitest during CI/test boot. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 7b8a9c8. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated CI to run tests in parallel shards (generators and unit), improving feedback speed and artifact reporting. * **Bug Fixes** * Stabilized test boot process to prevent interfering test runners during spec runs, improving test reliability. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
Summary
rspec-package-testsCI job into two parallel matrix shards: generators (429 examples with heavy filesystem I/O) and unit (584 examples covering everything else)Expected CI time reduction: ~24 min → ~12 min.
Closes #3131
Test plan
generatorsandunitshard jobs appear in the CI matrixspec/react_on_rails/generatorsspec/react_on_railsexcluding generators🤖 Generated with Claude Code
Note
Medium Risk
Medium risk because it changes CI execution by splitting the RSpec suite into shard-specific commands and adjusts test initialization to suppress Minitest/Test::Unit autorun behavior, which could affect what tests run and how failures surface.
Overview
Splits
rspec-package-testsinto two parallel matrix shards (generatorsvsunit) for both PR and full (main/full-ci) runs, and runs shard-specific RSpec commands (generators-only vs everything excluding generators).Updates uploaded artifact names to include the shard to avoid collisions, and tweaks both
spec_helper.rbfiles torequire "minitest"and disable Minitest/Test::Unit autorun hooks so RSpec CLI args don’t get parsed by Minitest during CI/test boot.Reviewed by Cursor Bugbot for commit 7b8a9c8. Bugbot is set up for automated code reviews on this repo. Configure here.
Summary by CodeRabbit