Repository navigation
fix(rust): match managed rg globs under ./-prefixed search paths - #229
Conversation
|
Hi, just checking in. This still merges cleanly against main. Happy to rework it if you'd prefer a different approach. |
|
Reviewed PR head The original bug is still present on main. The focused fix at the walker boundary looks appropriate for the original baseline. However, two changes are needed before integrating it with current main:
Both findings concern integration with changes made on main after this PR's baseline. These conclusions are based on static inspection of the source and pinned dependencies; I did not compile or run tests. |
Managed rg resolved relative search paths with root.join(path), so `.`, `./` and `./src` walked `<root>/./...`. The ignore crate strips the root from candidates as a byte prefix and matched `./src/a.rs`, so rules with a slash never matched: `-g 'src/**' needle .` returned nothing, and `-g '!docs/**'` or an --ignore-file pattern such as `docs/*.rs` excluded nothing. Drop `.` segments from relative walk roots, matching what ripgrep sees from the workspace root.
Dropping `.` segments with components() also dropped a trailing `/` or `/.`, so `zg --rg --files src/keep.rs/` listed the file and exited 0 where main and ripgrep report "Not a directory". Put the trailing marker back when the path names something below the root, and update the regression test for the relative_path() accessor from zvec-ai#287.
bf43918 to
c790f2a
Compare
|
Thanks for checking it against current main. Rebased onto 891497f and pushed c790f2a for both points:
Two new tests in I also added
|
|
Reviewed the updated head Both previous findings are addressed: the search test now uses One remaining edge-case regression: [P2] Preserve the workspace root's prefix when normalizing relative inputs. At On Unix, For example, given an existing Unix workspace with {"root":"//tmp/workspace","command":"rg --files -g 'src/**' src"}Please normalize only the relative input while preserving the workspace root's prefix, and add coverage for a double-slash root with include and exclude rules. This keeps the change focused on removing redundant This review is based on static inspection of the source and pinned dependencies; I did not compile or run tests. |
ZG Retrieval BenchmarkRetrieval-only · PR base vs head✅ Complete Base: zvec-ai/zvec-grep@891497fa87d3e03d0c9273fed0ba3e75af62f179 Head: JayOfTheKeyboard/zvec-grep@bdfe27e1e57746bee2e930e0a9a79afa6b0dc7ed Each cell is base / head / Δ. Δ = head − base; all five metrics use the 0–1 scale and higher is better. Both versions use the same frozen harness, queries, gold labels and local embedding models, with fresh indexes and five calls per query/mode. Quality is report-only; no regression threshold is applied.
SWE-QA nDCG uses repository macro averaging; the other suites use query means. Suites are reported separately. Full base/head reports and per-call evidence are available in the run artifacts. Embedding: local. e2e: skipped. This comment is updated by automatic PR Retrieval runs. |
…rg inputs Only the relative input is normalized, so a root such as //tmp/workspace keeps its prefix and still matches the override root that the ignore crate strips byte for byte. Include and exclude path globs work again under such a root.
|
Good catch, thanks. Pushed bdfe27e: New test |
Problem
In the Rust
zg, managed--rg(and the MCPzvec_grep_rgtool, which calls the same engine method) ignores every glob or--ignore-filerule that contains a/when the search path starts with.. Agents append.torgcommands all the time. Include globs then return nothing, and exclusions are silently dropped.Measured on a 207-file Go repository, with
zgbuilt frommain(28ef200) and compared against ripgrep 14.1.1. Each row counts matchingpath:linepairs.zgonmainrgzgwith this PR--rg -g 'internal/**' budget(no path)--rg -g 'internal/**' budget .--rg -g 'internal/**' budget ./--rg -g 'internal/**' budget ./internal--rg -g 'internal/policy/*.go' budget .--rg -g '!internal/**' budget .--rg -g '!/internal/**' budget .--rg --ignore-file x.ignore budget .(file holdsinternal/policy/**)--rg --ignore-file x.ignore budget(no path)Over MCP (
zg --server --stdio --mcp-toolset full):With this PR, the second call returns the same results as the first.
Globs without a slash (
*.go,!*.md) are unaffected. So are paths written without./(internal,internal/) and the workspace's own.gitignorefiles.Cause
build_walkerresolves each search path withresolve_path, that isroot.join(path), so.becomes<root>/.and walk entries look like<root>/./internal/a.go. The override matcher is built withOverrideBuilder::new(root), andignore'sGitignore::stripremoves the root as a byte prefix and then one leading/. What reaches the glob is therefore./internal/a.go, whichinternal/**does not match. Rules from--ignore-filefail the same way. ripgrep never hits this, because it walks./internalfrom the current directory andstripdrops a leading./. The TypeScript implementation passes the paths torgunchanged, withcwd: root, so it matches ripgrep too.Change
Search paths given to the walker go through a new
resolve_match_path, which drops.segments from the relative path and joins the result onto the root as given. The root itself is not normalised, so a root spelled//tmp/workspace(whichstd::path::absolutekeeps on Unix) still matches the root thatOverrideBuilderstrips. It puts a trailing/or/.back when the path names something below the root, sozg --rg --files src/keep.rs/still fails withNot a directory, as on main.resolve_pathitself is unchanged, socheck_paths,is_single_file_search, and the loading of pattern and ignore files behave as before.rg --filesgoes through the same walker and gets the same fix: on main,zg --rg --files -g 'src/**' .prints nothing.Not changed
rg -g 'internal/**' budget "$PWD/."also matches nothing in ripgrep 14.1.1 (and!internal/**excludes nothing), so normalising absolute paths would makezgdiffer fromrg. Happy to normalise those too if you would rather fix it than match it.resolve_path. A./prefix on the ignore file (--ignore-file ./x.ignore) was never the problem, and the unit test passes with or without it. Only the search path matters.zg <query> -g ...) do not take search paths and are unaffected. I checked-g 'internal/**'and-g '!internal/**'against an index of the same repository.Test
path_rules_match_search_paths_with_a_current_directory_prefixinzg-engine/src/lexical/mod.rsruns nine combinations of search path and glob or--ignore-fileoversrc/keep.rsanddocs/drop.rs. It collects every mismatch and expects onlysrc/keep.rseach time. The two cases with no path are controls that pass onmain.On the parent commit, the other seven cases fail (
./.withsrc/**was added in the second commit and fails the same way):Two tests in
zg-engine/src/lexical/files.rscoverrg --files:path_globs_match_inputs_with_a_current_directory_prefix(-g 'src/**'over.,./and./src/; fails on main) andfile_inputs_spelled_as_directories_report_errors(Unix only;src/keep.rs/,src/keep.rs/.and their./forms each give oneNot a directoryerror and no paths; passes on main, failed on this PR's first commit).path_globs_match_inputs_under_a_double_slash_root(Unix only) listssrcand./srcunder a//-prefixed root with-g 'src/**'and-g '!src/**'. It fails on the second commit, which normalised the root too (src src/**: []), and passes now. Over MCP,zvec_grep_rg {"root": "//<workspace>", "command": "rg --files -g 'src/**' src"}returned nothing on the second commit andsrc/keep.rsnow; with!src/**it listedsrc/keep.rsbefore and nothing now.Mutation check: removing the
.filtering fails the same cases, normalising the whole joined path again fails the double-slash test, and so does pointingbuild_walkerback atresolve_path. Removing either trailing-marker branch, or the check that the path names something below the root, fails a test.Validation
cargo fmt --all --check,cargo clippy --workspace --all-targets -- -D warnings, andRUSTDOCFLAGS="-D warnings" cargo doc -p zg-engine --no-depspass.cargo test -p zg-engine --lib lexical::passes (21 tests), and so doescargo test -p zg-engine --lib(663 passed, 9 ignored), on 891497f.cargo test -p zg-engine --lib663 passed, 1 failed, 9 ignored. The failure wasmissing_graph_is_created_without_rebuilding_indexed_files, which passes on its own.cargo clippy -p zg-engine --all-targets -- -D warningspasses. Thezg-engineintegration tests that use the mock embedding server fail here for the reason below, and fail on the previous head too.cargo test --workspace --no-fail-fast(final state): 801 passed, 20 failed, 8 ignored. Every failure is in a test that runs a local mock HTTP server or the daemon. The same group fails onmainon this machine (795 passed, 26 failed), and which of those tests fail changes from run to run. Another process here probes newly opened loopback ports with a Go HTTP client, and the mocks reject its request (user-agent: go-http-client/1.1, noauthorizationheader). The daemon test that exercises managed rg,full_toolset_exposes_lifecycle_tools_and_runs_managed_rg, passed 3 of 3 runs on its own on this branch, and 2 of 3 onmain. I'm relying on CI for this group.path:linesets, not only the counts.