Skip to content

Shard CI tests across runners and add one required CI check - #915

Merged
sds merged 3 commits into
mainfrom
ci/shard-tests
Nov 5, 2026
Merged

sds merged 3 commits into
mainfrom
ci/shard-tests

Conversation

@sds

@sds sds commented Nov 5, 2026 •

Copy link
Copy Markdown
Owner

Split each supported Ruby/OS combination across three GitHub runners. CI benchmarks improve complete workflow time from 4m 08s to 2m 15s (46% faster); two shards took 2m 51s. Each runner retains the existing local worker pool.

The shard selector deterministically assigns every example to exactly one runner. Sixteen new examples cover partition completeness, balance, ordering, and invalid shard settings. Each runner still verifies its executed example IDs and merges local worker coverage; unique Coveralls flags preserve coverage across all shards.

Add a single final check named CI for repository rulesets. It depends on every test shard and lint, explicitly fails for failed/canceled/skipped dependencies, and finalizes coverage reporting. Move lint into the same workflow so the required check includes it. Require CI instead of individual Ruby/OS checks.

GitHub Actions benchmarks (complete workflow, including setup and reporting):

Shards per Ruby/OS combination Matrix jobs Elapsed
1 (baseline) 6 4m 08s
2 12 2m 51s
3 (benchmark) 18 2m 20s
3 (final PR verification) 18 2m 15s

Validation: both benchmark workflows and final PR-head CI pass, including all 18 shards, lint, and final coverage reporting. Logs verify all 10,170 example executions (1,695 on each of six Ruby/OS combinations), with consistent suite identities and complete shard coverage. Local two-shard execution passes all 1,695 examples (the existing 1,679 plus 16 new shard-selector tests). Actionlint and RuboCop pass. The required-check predicate rejects failure/cancellation/skipped results in exhaustive local truth-table checks. Final PR-head verification: successful run on ce3e508. Linux test steps take 9–19s; Windows test steps take 59–96s. No repository ruleset settings were changed; the check is ready to select as CI.

@sds
sds merged commit 653a990 into main Nov 5, 2026
39 checks passed
@sds
sds deleted the ci/shard-tests branch November 5, 2026 08:26
sds added a commit to guillaume-d/overcommit that referenced this pull request Nov 10, 2026
* origin/main: (108 commits)
  Remember all previously approved configurations and hooks (sds#665)
  Keep deleted and renamed paths absent while stashing changes (sds#928)
  Document the case-sensitive KtLint hook name (sds#927)
  Update CI actions and pin Ubuntu to clear annotations (sds#926)
  Preserve standard input bytes when forwarding hooks (sds#925)
  Preserve Git-quoted filenames when running all hooks (sds#924)
  Fail when Git cannot enumerate tracked files (sds#923)
  Add note about disabling default hooks (sds#922)
  Close files inspected by LineEndings (sds#921)
  Use Script Host before PowerShell for Windows parent query fallback (sds#920)
  Cache Windows dependency resolution with daily refreshes (sds#919)
  Avoid PowerShell startup for Windows parent command queries (sds#918)
  Avoid repeated CLI startup when preparing integration fixtures (sds#917)
  Allocate more CI runners to Windows with asymmetric sharding (sds#916)
  Shard CI tests across runners and add one required CI check (sds#915)
  Remove EOL Ruby versions from the CI matrix (sds#914)
  Cut CI test time by 68% while preserving the full test matrix (sds#913)
  Cut version 0.75.0 (sds#912)
  Re-enable the full Windows test matrix and fix platform failures (sds#911)
  Read .overcommit.yml as UTF-8 in the hook template (sds#910)
  Update Overcommit logo (sds#907)
  Cut version 0.74.0 (sds#906)
  Keep each worktree's signatures separate (sds#903)
  Recognize amend aliases on macOS (sds#904)
  Exclude files deleted from the working tree from all_files (sds#905)
  Restore the Ruby 2.6-3.1 test checks (sds#902)
  Delete subprocess output tempfiles once they are read (sds#901)
  Stop wrapping commands in cmd.exe on Windows (sds#898)
  Allow URL-only lines to exceed TextWidth body limit (sds#900)
  Cut version 0.73.0 (sds#896)
  Add automated release flow (sds#895)
  Instruct oxfmt not to fail if no files are matched (sds#894)
  Add filetypes to oxc tools (sds#893)
  Add pre commit hook for herb-lint (sds#892)
  Cut version 0.72.0 (sds#891)
  Remove maintainability badge (sds#890)
  Require patched REXML version (sds#889)
  Fix operation state restoration in linked worktrees (sds#887)
  Fail Flay hook when flay crashes instead of silently passing (sds#885)
  Fix Solargraph regexp capture warning (sds#886)
  Cut version 0.71.0 (sds#883)
  Prepare for the new solargraph typecheck format (sds#873)
  Make hook context input reads thread-safe (sds#882)
  Cut version 0.70.0 (sds#880)
  Add oxc pre-commit hooks (sds#879)
  Add Ruby 3.4 and 4.0 to CI matrix (sds#878)
  Cut version 0.69.0 (sds#877)
  Use Etc.nprocessors and handle sandboxed environments (sds#876)
  Cut version 0.68.0 (sds#872)
  Add solargraph pre-commit hook for typechecking (sds#871)
  Update RuboCop in Gemfile to silence a deprecation warning in build (sds#870)
  Cut version 0.67.1
  adds 'set' requirements to diff hook context. (sds#866)
  Add note about unsupported YAML features in `gemfile:` line in default.yml (sds#865)
  Cut version 0.67.0 (sds#864)
  Restore ability to specify "gemfile: false" in config (sds#863)
  Cut version 0.66.0 (sds#861)
  Add `-diff` cli option for running precommit hooks against diffs (sds#860)
  Cut version 0.65.0
  Load bundled gems on expected version (sds#859)
  Update changelog
  Bump rexml to >= 3.3.9 to resolve GHSA-2rxp-v6pw-ch6m (sds#857)
  Expand tildes(~) for hooksPath (sds#853)
  Fix spec compatibility with recent git versions (sds#854)
  Update changelog
  Cut version 0.64.0 (sds#852)
  Add changelog_uri to gemspec (sds#851)
  Support stylelint 16+, which emits messages on stderr (sds#848)
  Add Ruby 3.3 to CI matrix (sds#849)
  Update README with latest requirements (sds#845)
  Fix README badges (sds#844)
  Cut version 0.63.0 (sds#843)
  Add RSpec Hook to PreCommit, Add Applicable files if set (sds#842)
  Fix RuboCop warnings (sds#841)
  [FEATURE] New pre-commit hook: Sorbet (Issue sds#825) (sds#826)
  Green CI PoC (sds#833)
  Revert "Get suite running smoothly again (sds#824)" (sds#840)
  Get suite running smoothly again (sds#824)
  Cut version 0.62.0 (sds#821)
  Allow childprocess 5 (sds#820)
  Update branch for CI from `master` to `main` (sds#819)
  Update RuboCop (sds#818)
  Adding Ruby 3.2 to test matrix (sds#816)
  Cut version 0.61.0 (sds#817)
  fix: local overcommit file merges with existing .overcommit.yml file (sds#815)
  Allow ReplaceBranch to use skip_if (sds#809)
  Cut version 0.60.0 (sds#807)
  Add non ASCII encoding schema support to RailsSchemaUpToDate hook (sds#786)
  Rescue Psych::DisallowedClass in YamlSyntax (sds#805)
  Add optional additional local configuration (sds#799)
  Add MixTest hook (sds#802)
  Add MixFormat hook (sds#801)
  support trailing space in replace_branch's 'replacement_text' config (sds#804)
  Allow overriding Gemfile.lock (sds#796)
  Cut version 0.59.1 (sds#785)
  Revert addition of `--disable-pending-cops` flag to RuboCop (sds#783)
  Revert `Fix encoding of process output under windows` (sds#784)
  Cut version 0.59.0 (sds#780)
  Partial log flushing to make long hooks intuitive (sds#777)
  Fix pronto hooks swallowing generic errors (sds#779)
  ...

# Conflicts:
#	CHANGELOG.md
#	README.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant