Skip to content

Exclude files deleted from the working tree from all_files - #905

Merged
sds merged 2 commits into
sds:mainfrom
ybai08:fix-run-all-deleted-files
Oct 31, 2026
Merged

sds merged 2 commits into
sds:mainfrom
ybai08:fix-run-all-deleted-files

Conversation

@ybai08

@ybai08 ybai08 commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Refs #699: this fixes the overcommit --run case narrowed down in #699 (comment) and #699 (comment). The renamed-file behaviour in the original report (during a normal commit) is not changed here.

GitRepo.all_files returns everything git ls-files lists. That includes a tracked file that has been deleted from the working tree when the deletion is not staged yet. overcommit --run uses all_files as its list of modified files, so every hook that reads its applicable_files then fails on that path. With JsonSyntax enabled:

$ rm a.json      # tracked, deletion not staged
$ overcommit --run
Running pre-commit hooks
Validate JSON syntax.....................................[JsonSyntax] FAILED
Hook raised unexpected error
No such file or directory @ rb_sysopen - .../a.json

all_files now drops the paths git ls-files --deleted reports. This uses Git's own view rather than File.exist?, so a tracked broken symlink is still listed (the BrokenSymlinks hook needs it). included_files (used by the pre-push Minitest hook) goes through all_files too, so it no longer passes a deleted file either.

Tests

A new context in spec/overcommit/git_repo_spec.rb deletes a committed file without staging the deletion and checks that all_files no longer includes it but still includes the other files.

Run on macOS with Ruby 4.0.7 and Git 2.54.0:

  • bundle exec rspec spec/overcommit/git_repo_spec.rb -e '.all_files': 4 examples, 1 failure with lib/overcommit/git_repo.rb from main (the new "is expected not to include .../tracked" example), 4 examples, 0 failures with the change.
  • Manual check in a scratch repository with only JsonSyntax enabled, a committed a.json deleted from the working tree, then overcommit --run: main fails with No such file or directory @ rb_sysopen; with the change, [JsonSyntax] OK.
  • bundle exec rspec: 1659 examples, 4 failures. The same 4 examples fail on main: the git alias #amendment? examples in hook_context/commit_msg_spec.rb:134,140 and hook_context/pre_commit_spec.rb:43,49, which look specific to this machine's Git (Apple Git).
  • bundle exec rubocop: no offenses.

Not run: anything on Windows or Linux.

`overcommit --run` checks every file `git ls-files` lists, which
includes tracked files that were deleted but whose deletion is not
staged. Hooks then fail trying to read them (e.g. JsonSyntax raises
'No such file or directory'). Skip the files `git ls-files --deleted`
reports.
@ybai08

ybai08 commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

The four failing rspec jobs (Ruby 2.6, 2.7, 3.0, 3.1) fail in the "Set up Ruby" step, before any spec runs: Unavailable version 3.1.7 for ruby on ubuntu-26.04 (and the same for 2.6.10, 2.7.8 and 3.0.7). The runner image no longer offers those Rubies. The newer Ruby jobs pass.

@sds sds added the bug label Oct 31, 2026
@sds
sds merged commit c9f6b35 into sds:main Oct 31, 2026
19 checks passed
@sds

sds commented Oct 31, 2026

Copy link
Copy Markdown
Owner

Thanks

sds added a commit that referenced this pull request Nov 10, 2026
Older Git versions can resurrect staged deletions and old rename paths as untracked files while commit hooks run. The scripted `git stash --keep-index` resets the working tree to HEAD, then restores the index without removing those paths, so tools such as RSpec can discover obsolete files.

Stash normally and restore the staged tree with `git read-tree --reset -u stash@{0}^2`. This updates the index and working tree without moving HEAD. If restoring the staged tree fails, stop before running hooks and retain the stash for recovery. Only restore it when this run actually created a stash.

Fixes the original normal-commit report in #699; the separate `overcommit --run` case was addressed in #905.

The added examples exposed a coverage-reporting problem: round-robin shard assignments shift when tests are added, so per-shard coverage comparisons can fail despite higher total coverage. CI now collects every shard's LCOV report and submits them together for one complete-suite coverage comparison. All test matrix jobs still run, and missing coverage artifacts fail the build.

Validation:

- Full suite: 1,750 examples, zero failures.
- All pre-commit checks and RuboCop pass.
- Added 12 examples covering pre-commit and commit-message contexts, current and emulated legacy stash behavior, complete state restoration, existing stashes, and setup failure recovery. No existing tests were removed.
- Confirmed setup and cleanup against Git 2.20's actual stash script, with its legacy text-output helper restored for compatibility with current Git.
- Confirmed the old-name regression tests fail before the fix.

Fixes #699.
sds added a commit to guillaume-d/overcommit that referenced this pull request Nov 10, 2026
* origin/main: (108 commits)
  Remember all previously approved configurations and hooks (sds#665)
  Keep deleted and renamed paths absent while stashing changes (sds#928)
  Document the case-sensitive KtLint hook name (sds#927)
  Update CI actions and pin Ubuntu to clear annotations (sds#926)
  Preserve standard input bytes when forwarding hooks (sds#925)
  Preserve Git-quoted filenames when running all hooks (sds#924)
  Fail when Git cannot enumerate tracked files (sds#923)
  Add note about disabling default hooks (sds#922)
  Close files inspected by LineEndings (sds#921)
  Use Script Host before PowerShell for Windows parent query fallback (sds#920)
  Cache Windows dependency resolution with daily refreshes (sds#919)
  Avoid PowerShell startup for Windows parent command queries (sds#918)
  Avoid repeated CLI startup when preparing integration fixtures (sds#917)
  Allocate more CI runners to Windows with asymmetric sharding (sds#916)
  Shard CI tests across runners and add one required CI check (sds#915)
  Remove EOL Ruby versions from the CI matrix (sds#914)
  Cut CI test time by 68% while preserving the full test matrix (sds#913)
  Cut version 0.75.0 (sds#912)
  Re-enable the full Windows test matrix and fix platform failures (sds#911)
  Read .overcommit.yml as UTF-8 in the hook template (sds#910)
  Update Overcommit logo (sds#907)
  Cut version 0.74.0 (sds#906)
  Keep each worktree's signatures separate (sds#903)
  Recognize amend aliases on macOS (sds#904)
  Exclude files deleted from the working tree from all_files (sds#905)
  Restore the Ruby 2.6-3.1 test checks (sds#902)
  Delete subprocess output tempfiles once they are read (sds#901)
  Stop wrapping commands in cmd.exe on Windows (sds#898)
  Allow URL-only lines to exceed TextWidth body limit (sds#900)
  Cut version 0.73.0 (sds#896)
  Add automated release flow (sds#895)
  Instruct oxfmt not to fail if no files are matched (sds#894)
  Add filetypes to oxc tools (sds#893)
  Add pre commit hook for herb-lint (sds#892)
  Cut version 0.72.0 (sds#891)
  Remove maintainability badge (sds#890)
  Require patched REXML version (sds#889)
  Fix operation state restoration in linked worktrees (sds#887)
  Fail Flay hook when flay crashes instead of silently passing (sds#885)
  Fix Solargraph regexp capture warning (sds#886)
  Cut version 0.71.0 (sds#883)
  Prepare for the new solargraph typecheck format (sds#873)
  Make hook context input reads thread-safe (sds#882)
  Cut version 0.70.0 (sds#880)
  Add oxc pre-commit hooks (sds#879)
  Add Ruby 3.4 and 4.0 to CI matrix (sds#878)
  Cut version 0.69.0 (sds#877)
  Use Etc.nprocessors and handle sandboxed environments (sds#876)
  Cut version 0.68.0 (sds#872)
  Add solargraph pre-commit hook for typechecking (sds#871)
  Update RuboCop in Gemfile to silence a deprecation warning in build (sds#870)
  Cut version 0.67.1
  adds 'set' requirements to diff hook context. (sds#866)
  Add note about unsupported YAML features in `gemfile:` line in default.yml (sds#865)
  Cut version 0.67.0 (sds#864)
  Restore ability to specify "gemfile: false" in config (sds#863)
  Cut version 0.66.0 (sds#861)
  Add `-diff` cli option for running precommit hooks against diffs (sds#860)
  Cut version 0.65.0
  Load bundled gems on expected version (sds#859)
  Update changelog
  Bump rexml to >= 3.3.9 to resolve GHSA-2rxp-v6pw-ch6m (sds#857)
  Expand tildes(~) for hooksPath (sds#853)
  Fix spec compatibility with recent git versions (sds#854)
  Update changelog
  Cut version 0.64.0 (sds#852)
  Add changelog_uri to gemspec (sds#851)
  Support stylelint 16+, which emits messages on stderr (sds#848)
  Add Ruby 3.3 to CI matrix (sds#849)
  Update README with latest requirements (sds#845)
  Fix README badges (sds#844)
  Cut version 0.63.0 (sds#843)
  Add RSpec Hook to PreCommit, Add Applicable files if set (sds#842)
  Fix RuboCop warnings (sds#841)
  [FEATURE] New pre-commit hook: Sorbet (Issue sds#825) (sds#826)
  Green CI PoC (sds#833)
  Revert "Get suite running smoothly again (sds#824)" (sds#840)
  Get suite running smoothly again (sds#824)
  Cut version 0.62.0 (sds#821)
  Allow childprocess 5 (sds#820)
  Update branch for CI from `master` to `main` (sds#819)
  Update RuboCop (sds#818)
  Adding Ruby 3.2 to test matrix (sds#816)
  Cut version 0.61.0 (sds#817)
  fix: local overcommit file merges with existing .overcommit.yml file (sds#815)
  Allow ReplaceBranch to use skip_if (sds#809)
  Cut version 0.60.0 (sds#807)
  Add non ASCII encoding schema support to RailsSchemaUpToDate hook (sds#786)
  Rescue Psych::DisallowedClass in YamlSyntax (sds#805)
  Add optional additional local configuration (sds#799)
  Add MixTest hook (sds#802)
  Add MixFormat hook (sds#801)
  support trailing space in replace_branch's 'replacement_text' config (sds#804)
  Allow overriding Gemfile.lock (sds#796)
  Cut version 0.59.1 (sds#785)
  Revert addition of `--disable-pending-cops` flag to RuboCop (sds#783)
  Revert `Fix encoding of process output under windows` (sds#784)
  Cut version 0.59.0 (sds#780)
  Partial log flushing to make long hooks intuitive (sds#777)
  Fix pronto hooks swallowing generic errors (sds#779)
  ...

# Conflicts:
#	CHANGELOG.md
#	README.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants