Repository navigation
fix(discover): stack every nested .gitignore on a path, not just the shallowest (#1973) - #2095
Merged
Merged
Conversation
…shallowest try_load_nested_gitignore() returned NULL as soon as a walk frame already carried a local matcher, and that matcher was inherited by every child frame. So only the SHALLOWEST nested .gitignore on a path was ever loaded; any deeper one was silently dropped — an EMPTY storage/.gitignore was enough to disable storage/dump/.gitignore's "*". On the standard Laravel layout (storage/.gitignore above storage/<dir>/.gitignore) that meant thousands of git-ignored cache/debugbar dumps were discovered and indexed, which on the reporter's app grew to 25,638 discovered files against 6,682 git-visible ones and ended in an OOM kill (#1973, discovery half; also the §2 root cause of #530). Discovery now keeps a root->leaf chain of .gitignore links per directory frame: every directory below the root loads its own .gitignore, a frame without one shares its parent's link, and a path is judged by walking the chain leaf->root with git's precedence — the deepest file with an opinion wins (so a deeper negation re-includes what an ancestor ignores), and inside one file the last matching pattern wins. The repository root's .gitignore (merged with info/exclude) is the chain head, so the same precedence holds between root and nested files. Cost is one match per .gitignore on the path — O(depth) per entry, no rescans; the walk frame shrinks by the 4 KB local-prefix buffer it no longer needs. Tests pin the reporter's layout (empty ancestor + deeper "*"), a deeper negation overriding an ancestor pattern, and three nesting levels with an empty middle file. Measured on a Laravel app (koel) seeded with typical compiled-view/cache artifacts: 2,885 discovered files with 450 git-ignored -> 2,435 with 0; clean clones of linux, dotnet/runtime, koel and redis discover byte-identical file sets before and after. Fixes the discovery half of #1973. Co-Authored-By: Claude Fable 5.1 <[email protected]> Signed-off-by: Martin Vogel <[email protected]>
Open
2 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the discovery half of #1973 (the memory-budget half is a separate change).
Only the shallowest nested
.gitignoreon a path was ever loaded:try_load_nested_gitignore()bailed when the frame already carried a local matcher, and that matcher was inherited by children. An emptystorage/.gitignoretherefore disabledstorage/dump/.gitignore's*, so a Laravel tree's dumps, caches and sessions were discovered and indexed (the reporter's synthetic tree: 5000 ignored JSON files → 1,020,008 nodes; the real app: 25,638 discovered vs 6,682 git-visible, then OOM).Fix (
src/discover/discover.c): each directory frame carries a root→leaf chain of heap-owned (matcher, prefix) links (the root.gitignoreincl.info/excludeis the chain head; a directory without its own file shares the parent link). A path is judged leaf→root and the first decisive verdict wins, matching git (a lower-level file takes precedence; last match within a file). O(number of.gitignorefiles on the path) per entry, no rescans.Behaviour change: a nested
.gitignorenegation now re-includes what the root.gitignoreignores (git-correct; previously root always won).Tests (
tests/test_discover.c): the reporter's shape (rootnode_modules/, emptystorage/.gitignore,storage/dump/.gitignore*) yields onlyapp/x.phpand a bounded count of 1; a deeper negation overrides an ancestor pattern; three nesting levels with an empty middle file. RED on unmodified production (3 failures), GREEN with the fix, RED again on revert.Local verification (macOS host): discover 117/0, pipeline 273/0, complexity 5/0 (the O(n²) guard stays at ratio 2), bug-repro
repro_issue510_nested_gitignore_honoredPASS,make lint-ciclean. Discovery on clean clones is byte-identical before/after (linux 89,767 files with 403.gitignorefiles, dotnet/runtime 42,555, koel 2,435, redis 1,750); a koel copy seeded with 450 typical git-ignored Laravel artifacts drops from 2,885 discovered (450 ignored by git) to 2,435 (0).Pre-existing, recorded: a
.gitignorenegation still does not beat the globalcore.excludesFilematcher, and.git/info/excludecarries root precedence rather than git's lowest.