Skip to content

fix(rust): match managed rg globs under ./-prefixed search paths - #229

Merged
egolearner merged 3 commits into
zvec-ai:mainfrom
JayOfTheKeyboard:fix/rg-dot-path-globs
Oct 30, 2026
Merged

egolearner merged 3 commits into
zvec-ai:mainfrom
JayOfTheKeyboard:fix/rg-dot-path-globs

Conversation

@JayOfTheKeyboard

@JayOfTheKeyboard JayOfTheKeyboard commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Problem

In the Rust zg, managed --rg (and the MCP zvec_grep_rg tool, which calls the same engine method) ignores every glob or --ignore-file rule that contains a / when the search path starts with .. Agents append . to rg commands all the time. Include globs then return nothing, and exclusions are silently dropped.

Measured on a 207-file Go repository, with zg built from main (28ef200) and compared against ripgrep 14.1.1. Each row counts matching path:line pairs.

Command zg on main rg zg with this PR
--rg -g 'internal/**' budget (no path) 167 167 167
--rg -g 'internal/**' budget . 0 167 167
--rg -g 'internal/**' budget ./ 0 167 167
--rg -g 'internal/**' budget ./internal 0 167 167
--rg -g 'internal/policy/*.go' budget . 0 35 35
--rg -g '!internal/**' budget . 252 (nothing excluded) 85 85
--rg -g '!/internal/**' budget . 252 85 85
--rg --ignore-file x.ignore budget . (file holds internal/policy/**) 252 217 217
--rg --ignore-file x.ignore budget (no path) 217 217 217

Over MCP (zg --server --stdio --mcp-toolset full):

zvec_grep_rg {"command": "rg -g 'internal/**' budget"}    -> internal/e2e/counter_split_test.go ...
zvec_grep_rg {"command": "rg -g 'internal/**' budget ."}  -> No matches.

With this PR, the second call returns the same results as the first.

Globs without a slash (*.go, !*.md) are unaffected. So are paths written without ./ (internal, internal/) and the workspace's own .gitignore files.

Cause

build_walker resolves each search path with resolve_path, that is root.join(path), so . becomes <root>/. and walk entries look like <root>/./internal/a.go. The override matcher is built with OverrideBuilder::new(root), and ignore's Gitignore::strip removes the root as a byte prefix and then one leading /. What reaches the glob is therefore ./internal/a.go, which internal/** does not match. Rules from --ignore-file fail the same way. ripgrep never hits this, because it walks ./internal from the current directory and strip drops a leading ./. The TypeScript implementation passes the paths to rg unchanged, with cwd: root, so it matches ripgrep too.

Change

Search paths given to the walker go through a new resolve_match_path, which drops . segments from the relative path and joins the result onto the root as given. The root itself is not normalised, so a root spelled //tmp/workspace (which std::path::absolute keeps on Unix) still matches the root that OverrideBuilder strips. It puts a trailing / or /. back when the path names something below the root, so zg --rg --files src/keep.rs/ still fails with Not a directory, as on main. resolve_path itself is unchanged, so check_paths, is_single_file_search, and the loading of pattern and ignore files behave as before. rg --files goes through the same walker and gets the same fix: on main, zg --rg --files -g 'src/**' . prints nothing.

Not changed

  • Absolute paths are kept as given. rg -g 'internal/**' budget "$PWD/." also matches nothing in ripgrep 14.1.1 (and !internal/** excludes nothing), so normalising absolute paths would make zg differ from rg. Happy to normalise those too if you would rather fix it than match it.
  • The ignore-file path itself is still resolved with resolve_path. A ./ prefix on the ignore file (--ignore-file ./x.ignore) was never the problem, and the unit test passes with or without it. Only the search path matters.
  • Indexed search globs (zg <query> -g ...) do not take search paths and are unaffected. I checked -g 'internal/**' and -g '!internal/**' against an index of the same repository.

Test

path_rules_match_search_paths_with_a_current_directory_prefix in zg-engine/src/lexical/mod.rs runs nine combinations of search path and glob or --ignore-file over src/keep.rs and docs/drop.rs. It collects every mismatch and expects only src/keep.rs each time. The two cases with no path are controls that pass on main.

On the parent commit, the other seven cases fail (./. with src/** was added in the second commit and fails the same way):

"paths [\".\"] src/**: []",
"paths [\"./\"] src/**: []",
"paths [\"./src\"] src/*.rs: []",
"paths [\".\"] !docs/**: [\"docs/drop.rs\", \"src/keep.rs\"]",
"paths [\"./docs\", \"./src\"] !docs/**: [\"docs/drop.rs\", \"src/keep.rs\"]",
"paths [\".\"] --ignore-file custom.ignore: [\"docs/drop.rs\", \"src/keep.rs\"]",

Two tests in zg-engine/src/lexical/files.rs cover rg --files: path_globs_match_inputs_with_a_current_directory_prefix (-g 'src/**' over ., ./ and ./src/; fails on main) and file_inputs_spelled_as_directories_report_errors (Unix only; src/keep.rs/, src/keep.rs/. and their ./ forms each give one Not a directory error and no paths; passes on main, failed on this PR's first commit).

path_globs_match_inputs_under_a_double_slash_root (Unix only) lists src and ./src under a //-prefixed root with -g 'src/**' and -g '!src/**'. It fails on the second commit, which normalised the root too (src src/**: []), and passes now. Over MCP, zvec_grep_rg {"root": "//<workspace>", "command": "rg --files -g 'src/**' src"} returned nothing on the second commit and src/keep.rs now; with !src/** it listed src/keep.rs before and nothing now.

Mutation check: removing the . filtering fails the same cases, normalising the whole joined path again fails the double-slash test, and so does pointing build_walker back at resolve_path. Removing either trailing-marker branch, or the check that the path names something below the root, fails a test.

Validation

  • cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings, and RUSTDOCFLAGS="-D warnings" cargo doc -p zg-engine --no-deps pass.
  • cargo test -p zg-engine --lib lexical:: passes (21 tests), and so does cargo test -p zg-engine --lib (663 passed, 9 ignored), on 891497f.
  • Third commit (bdfe27e): cargo test -p zg-engine --lib 663 passed, 1 failed, 9 ignored. The failure was missing_graph_is_created_without_rebuilding_indexed_files, which passes on its own. cargo clippy -p zg-engine --all-targets -- -D warnings passes. The zg-engine integration tests that use the mock embedding server fail here for the reason below, and fail on the previous head too.
  • cargo test --workspace --no-fail-fast (final state): 801 passed, 20 failed, 8 ignored. Every failure is in a test that runs a local mock HTTP server or the daemon. The same group fails on main on this machine (795 passed, 26 failed), and which of those tests fail changes from run to run. Another process here probes newly opened loopback ports with a Go HTTP client, and the mocks reject its request (user-agent: go-http-client/1.1, no authorization header). The daemon test that exercises managed rg, full_toolset_exposes_lifecycle_tools_and_runs_managed_rg, passed 3 of 3 runs on its own on this branch, and 2 of 3 on main. I'm relying on CI for this group.
  • Real-tool run: the table above was produced with a debug build before and after this change, against ripgrep 14.1.1 on the same tree, comparing the full path:line sets, not only the counts.

@JayOfTheKeyboard

Copy link
Copy Markdown
Contributor Author

Hi, just checking in. This still merges cleanly against main. Happy to rework it if you'd prefer a different approach.

Copy link
Copy Markdown
Contributor

Reviewed PR head bf4391810e against the fetched origin/main at 6e71bbb46f9.

The original bug is still present on main. build_walker still resolves search paths through root.join(path). On Unix, the pinned ignore 0.4.33 strips the workspace root as a byte prefix, leaving ./src/... for inputs such as . or ./src. Directory-qualified include/exclude globs and explicit ignore-file rules therefore still encounter the mismatch described here.

The focused fix at the walker boundary looks appropriate for the original baseline. However, two changes are needed before integrating it with current main:

  1. [P1] Update the regression test for the current match API. The new test accesses item.relative_path.clone(), but perf(rust): streamline rg search and reduce match allocations #287 replaced that field with the relative_path() accessor. The test target will fail to compile after merging with current main. Please use item.relative_path().to_path_buf().

  2. [P2] Preserve trailing-slash errors for file discovery. components().collect() also removes trailing / and /.. The newer file-discovery caller shares build_walker but deliberately bypasses check_paths, relying on the walker to report invalid inputs. If src/keep.rs is a regular file, zg --rg --files src/keep.rs/ (or src/keep.rs/.) should report a traversal error on Unix. This normalization instead turns the input into a valid file path; discovery then returns the original invalid spelling and exits successfully. This breaks the stated intent to preserve trailing-slash error behavior. Please retain that directory constraint, or validate the original discovery input while preserving its errors, and add a regression case.

Both findings concern integration with changes made on main after this PR's baseline. These conclusions are based on static inspection of the source and pinned dependencies; I did not compile or run tests.

Managed rg resolved relative search paths with root.join(path), so `.`,
`./` and `./src` walked `<root>/./...`. The ignore crate strips the root
from candidates as a byte prefix and matched `./src/a.rs`, so rules with
a slash never matched: `-g 'src/**' needle .` returned nothing, and
`-g '!docs/**'` or an --ignore-file pattern such as `docs/*.rs` excluded
nothing. Drop `.` segments from relative walk roots, matching what
ripgrep sees from the workspace root.
Dropping `.` segments with components() also dropped a trailing `/` or
`/.`, so `zg --rg --files src/keep.rs/` listed the file and exited 0
where main and ripgrep report "Not a directory". Put the trailing marker
back when the path names something below the root, and update the
regression test for the relative_path() accessor from zvec-ai#287.
@JayOfTheKeyboard

Copy link
Copy Markdown
Contributor Author

Thanks for checking it against current main. Rebased onto 891497f and pushed c790f2a for both points:

  • The test now uses item.relative_path().to_path_buf().
  • Trailing slash: you were right. A zg built from the previous head printed src/keep.rs/ for zg --rg --files src/keep.rs/ and exited 0, and did the same for src/keep.rs/., where main exits 2 with Not a directory. resolve_match_path still drops the . segments, but now puts the trailing / or /. back when the path names something below the root. Both inputs exit 2 again with main's error text (with a ./ prefix, the error shows the path without it).

Two new tests in lexical/files.rs. file_inputs_spelled_as_directories_report_errors (Unix only) checks that src/keep.rs/, src/keep.rs/., ./src/keep.rs/ and ./src/keep.rs/. each give one error and no paths; it fails on the previous head and passes on main. path_globs_match_inputs_with_a_current_directory_prefix runs --files -g 'src/**' over ., ./ and ./src/, since file discovery uses the same walker and had the same glob bug (on main, zg --rg --files -g 'src/**' . prints nothing and exits 1); it fails on main.

I also added ./. to the search test. Without the "names something below the root" check it kept its trailing /. and the walker was back at <root>/./src/.... Removing any part of the new check fails a test.

cargo fmt --check, cargo clippy --workspace --all-targets -- -D warnings and cargo test -p zg-engine --lib (663 passed) pass locally.

Copy link
Copy Markdown
Contributor

Reviewed the updated head c790f2a8ac633f8f9009abb57385e5e0238eea1d against base 891497fa87d3e03d0c9273fed0ba3e75af62f179.

Both previous findings are addressed: the search test now uses relative_path(), and file discovery preserves errors for regular-file inputs ending in / or /., with regression coverage. Keeping the fix at the shared walker boundary is appropriate.

One remaining edge-case regression:

[P2] Preserve the workspace root's prefix when normalizing relative inputs. At resolve_match_path, line 889, root.join(path).components().collect() normalizes the absolute workspace root as well as the relative search path.

On Unix, std::path::absolute deliberately preserves exactly two leading slashes. An MCP/API root such as //tmp/workspace therefore reaches this helper unchanged. For a relative input src, the previous implementation produced candidates such as //tmp/workspace/src/keep.rs, which matched src/**. The new implementation produces /tmp/workspace/src/keep.rs, while OverrideBuilder::new(root) still uses //tmp/workspace. The pinned ignore implementation strips that prefix byte-for-byte on Unix, so stripping now fails: directory-qualified include globs miss files, and corresponding exclude globs no longer exclude them.

For example, given an existing Unix workspace with src/keep.rs, this MCP request exercises the affected path:

{"root":"//tmp/workspace","command":"rg --files -g 'src/**' src"}

Please normalize only the relative input while preserving the workspace root's prefix, and add coverage for a double-slash root with include and exclude rules. This keeps the change focused on removing redundant . segments without breaking previously working inputs.

This review is based on static inspection of the source and pinned dependencies; I did not compile or run tests.

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

ZG Retrieval Benchmark

Retrieval-only · PR base vs head

✅ Complete

Base: zvec-ai/zvec-grep@891497fa87d3e03d0c9273fed0ba3e75af62f179

Head: JayOfTheKeyboard/zvec-grep@bdfe27e1e57746bee2e930e0a9a79afa6b0dc7ed

Each cell is base / head / Δ. Δ = head − base; all five metrics use the 0–1 scale and higher is better. Both versions use the same frozen harness, queries, gold labels and local embedding models, with fresh indexes and five calls per query/mode. Quality is report-only; no regression threshold is applied.

Suite Mode File Hit@1 File Hit@5 File Hit@10 MRR@10 nDCG@10
SWE-QA20 hybrid 0.3500 / 0.3500 / 0.0000 0.6000 / 0.6000 / 0.0000 0.7500 / 0.7500 / 0.0000 0.4710 / 0.4710 / 0.0000 0.4268 / 0.4268 / 0.0000
SWE-QA20 fts 0.4000 / 0.4000 / 0.0000 0.6500 / 0.6500 / 0.0000 0.8500 / 0.8500 / 0.0000 0.5007 / 0.5007 / 0.0000 0.4070 / 0.4070 / 0.0000
SWE-QA20 vector 0.3000 / 0.3000 / 0.0000 0.6500 / 0.6500 / 0.0000 0.7000 / 0.7000 / 0.0000 0.4655 / 0.4655 / 0.0000 0.4238 / 0.4238 / 0.0000
BEIR / four datasets hybrid 0.3500 / 0.3500 / 0.0000 0.6500 / 0.6500 / 0.0000 0.7000 / 0.7000 / 0.0000 0.5056 / 0.5056 / 0.0000 0.4784 / 0.4784 / 0.0000
BEIR / four datasets fts 0.5500 / 0.5500 / 0.0000 0.8000 / 0.8000 / 0.0000 0.8500 / 0.8500 / 0.0000 0.6633 / 0.6633 / 0.0000 0.5879 / 0.5879 / 0.0000
BEIR / four datasets vector 0.2000 / 0.2000 / 0.0000 0.6000 / 0.6000 / 0.0000 0.6500 / 0.6500 / 0.0000 0.3221 / 0.3221 / 0.0000 0.3675 / 0.3675 / 0.0000
DuRetrieval / Chinese web search hybrid 0.6000 / 0.6000 / 0.0000 1.0000 / 1.0000 / 0.0000 1.0000 / 1.0000 / 0.0000 0.7583 / 0.7583 / 0.0000 0.5090 / 0.5090 / 0.0000
DuRetrieval / Chinese web search fts 0.5000 / 0.5000 / 0.0000 0.9000 / 0.9000 / 0.0000 0.9000 / 0.9000 / 0.0000 0.6833 / 0.6833 / 0.0000 0.5589 / 0.5589 / 0.0000
DuRetrieval / Chinese web search vector 0.3000 / 0.3000 / 0.0000 0.9000 / 0.9000 / 0.0000 0.9000 / 0.9000 / 0.0000 0.4783 / 0.4783 / 0.0000 0.3872 / 0.3872 / 0.0000
Quarry / eight languages hybrid 0.6500 / 0.6500 / 0.0000 0.8000 / 0.8000 / 0.0000 0.8000 / 0.8000 / 0.0000 0.6950 / 0.6950 / 0.0000 0.6630 / 0.6630 / 0.0000
Quarry / eight languages fts 0.5500 / 0.5500 / 0.0000 0.7000 / 0.7000 / 0.0000 0.7500 / 0.7500 / 0.0000 0.6321 / 0.6321 / 0.0000 0.5940 / 0.5940 / 0.0000
Quarry / eight languages vector 0.7500 / 0.7500 / 0.0000 0.8000 / 0.8000 / 0.0000 0.9000 / 0.9000 / 0.0000 0.7794 / 0.7794 / 0.0000 0.7610 / 0.7610 / 0.0000

SWE-QA nDCG uses repository macro averaging; the other suites use query means. Suites are reported separately. Full base/head reports and per-call evidence are available in the run artifacts.

View run and artifacts

Embedding: local. e2e: skipped. This comment is updated by automatic PR Retrieval runs.

…rg inputs

Only the relative input is normalized, so a root such as //tmp/workspace
keeps its prefix and still matches the override root that the ignore crate
strips byte for byte. Include and exclude path globs work again under such
a root.
@JayOfTheKeyboard

Copy link
Copy Markdown
Contributor Author

Good catch, thanks. Pushed bdfe27e: resolve_match_path now drops the . segments from the relative input only and joins that onto the root as given, so //tmp/workspace keeps its prefix.

New test path_globs_match_inputs_under_a_double_slash_root in lexical/files.rs lists src and ./src under a // root with -g 'src/**' and -g '!src/**'. It fails on c790f2a and passes now. I also sent your MCP request through zvec_grep_rg with a // root: on c790f2a, rg --files -g 'src/**' src returned nothing and -g '!src/**' listed src/keep.rs; with the fix it's the other way round.

@egolearner
egolearner merged commit a8b7030 into zvec-ai:main Oct 30, 2026
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants