Skip to content

Fix multiline --count-matches totals across search backends - #192

Merged
Shengyu Fu (shengyfu) merged 2 commits into
mainfrom
shengyfu-fix-multiline-match-counting
Oct 11, 2026
Merged

Shengyu Fu (shengyfu) merged 2 commits into
mainfrom
shengyfu-fix-multiline-match-counting

Conversation

@shengyfu

@shengyfu Shengyu Fu (shengyfu) commented Oct 11, 2026 •

Copy link
Copy Markdown
Member

Fixes #188

Change

FileMatches already retained original regex-match totals after applying search limits, but --count-matches reconstructed its count from spans clipped onto individual output lines. One hello\nworld match therefore counted twice. The server client independently made the same mistake.

Use the original per-file totals for counting in scan/local-index searches and in ordinary/shared-v1/shared-v2 server responses. An opt-in count_matches request reuses the existing typed file_stats metadata without enabling --stats or index diagnostics. Count RPCs omit unused row spans and columns, without disabling local or server-side collection of all original spans. Preserve the existing -c, inversion, adjacent-match, JSON/Unicode offset, and other rendering behavior.

Non-counting requests retain their previous wire shape. Counting never guesses from old response rows: missing, malformed, partial, or duplicate per-file totals are rejected before any count output. Ordinary-server fallback includes the cause; older shared servers can reject the new option and use the existing diagnostic scan fallback. Older ordinary servers that provide totals with --stats remain usable.

Regression coverage

  • Exact stdout with and without --stats, default and PCRE2 engines, forced scan, local index, ordinary server, and shared v1/v2. Backend diagnostics are required; the shared-v1 query counter also proves service of the requests without stats.
  • Cross-line and repeated matches, independent per-file totals, shared/scoped paths, limits (including zero), inversion, independent/adjacent matches, Unicode, newline matches, replacement, and -o/--vimgrep combinations. The existing multiline stats test now asserts the actual count instead of only output parity.
  • Direct RPC coverage for totals without stats or row detail, legacy non-counting requests, and malformed/older responses with no partial output before fallback.
  • Request-construction coverage distinguishes count-only wire detail from internal span collection, including stats and output-mode precedence. Bounded, match-heavy real RPC replies from ordinary/shared-v1/shared-v2 servers must be below 40 KiB, contain no span/column arrays, and report exactly 32,768 or 32,769 original matches. Both engines and stats settings are exercised; legacy requests still return all spans. The shared checks require the actual ready backend and cannot pass by scanning.

The original matcher, scan, ordinary-RPC, and both shared regressions were observed failing before the count fix. The follow-up request regression separately failed before row-detail suppression (detail=true, expected false). Existing controls remain covered.

Validation

Build, Rayon, and test concurrency were bounded to two.

Review follow-up: 9f20f5ab6d1356ffa43aa30bfaa773b1ce4ca83c

Windows, Rust 1.98.0:

cargo test --locked -p tgrep-cli --bin tgrep --test ripgrep_compat --test shared_daemon count_matches -- --test-threads=2
cargo test --locked -p tgrep-cli --features managed-test-hooks --test shared_daemon count_matches --quiet -- --test-threads=2
cargo test --locked -p tgrep-cli --bin tgrep --test ripgrep_compat --test ripgrep_parity --test shared_daemon --no-fail-fast --quiet -- --test-threads=2
cargo fmt --all -- --check
cargo clippy --locked --workspace --all-targets --features tgrep-cli/managed-test-hooks --quiet -- -D warnings
  • Focused request/count/RPC regressions: 11 passed. Feature-enabled shared count regressions: 2 passed.
  • Affected suites: 703 passed, 3 existing ignored helpers (320 unit, 246 compatibility, 102 parity, 35 shared).
  • Formatting and strict Clippy passed. The final diff was directly reviewed; the worktree is clean.
  • This follow-up was not rerun locally on Linux or macOS; current-head native CI supplies cross-platform qualification. At publication those jobs and the new review were still in progress, not claimed passed. The original-head macOS job failed with shared request queue full in the unchanged runtime-attach fixture; this PR does not bypass that assertion or alter queue limits.

Original fix: 6a6440bdcc54f7416e490120738fc211daecf529

cargo test --locked -p tgrep-cli --bin tgrep --test ripgrep_compat --test ripgrep_parity --test shared_daemon --test concurrent_search --no-fail-fast -- --test-threads=2
cargo test --locked -p tgrep-cli --features managed-test-hooks --test shared_daemon multiline_count_matches -- --test-threads=2
cargo fmt --all -- --check
cargo clippy --locked --workspace --all-targets --features tgrep-cli/managed-test-hooks -- -D warnings
  • Windows, Rust 1.98.0: affected suites 710 passed, 3 existing ignored helpers; feature-enabled shared regressions 2 passed. Formatting and strict Clippy passed.
  • Native Linux filesystem under WSL, Rust 1.98.1: affected suites 736 passed, 3 existing ignored helpers; feature-enabled shared regressions 2 passed. Strict Clippy passed. The isolated snapshot's changed-file bytes were verified against the original head.
  • Independent acceptance reported 80/80 passing cases on the frozen candidate mapped to the original head: 48 scan/index/current-server cases, 16 against an older 1.2.0 server, and 16 against a historical 1.1.0 server. For both historical versions, combinations with --stats used the actual server; combinations without --stats explicitly fell back to the local index. The same verifier rejected the old client in the 12 expected cross-line cases while its 36 controls passed. These frozen-candidate results are not attributed to the follow-up head.

No dependencies, lockfiles, versions, release configuration, or index formats changed.

Count original per-file regex matches before clipping spans into output lines. Carry authoritative totals through ordinary and shared server responses without requiring --stats, and reject incomplete count metadata before output.

Co-authored-by: Copilot App <[email protected]>
Copilot AI balanced review requested due to automatic review settings October 11, 2026 15:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Count requests still transfer unused span detail, risking oversized shared responses and scan fallback.

1 open finding
What changed in this PR

Fixes multiline --count-matches by using original per-file match totals across local, ordinary-server, and shared-server backends.

Changes:

  • Uses authoritative match totals instead of rendered line spans.
  • Extends server requests/responses with optional count metadata.
  • Adds broad regression coverage and documentation.
File Description
README.md Documents multiline counting and fallback behavior.
tgrep-cli/​src/​matching.rs Returns original regex-match totals.
tgrep-cli/​src/​search.rs Requests, validates, and renders server totals.
tgrep-cli/​src/​serve.rs Produces per-file totals without enabling diagnostics.
tgrep-cli/​src/​serve/​shared/​server.rs Accepts the shared count option.
tgrep-cli/​tests/​ripgrep_compat.rs Covers local and ordinary-server behavior.
tgrep-cli/​tests/​shared_daemon.rs Covers shared-v1 counting.
tgrep-cli/​tests/​shared_daemon/​managed.rs Covers managed shared-v2 counting.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tgrep-cli/src/search.rs Resolved
Copilot AI balanced review requested due to automatic review settings October 11, 2026 15:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The implementation consistently uses validated original match totals and includes broad regression coverage across all affected backends.

0 open findings

1 resolved since last review

🧠 Review effort: Balanced

@shengyfu
Shengyu Fu (shengyfu) merged commit 669e832 into main Oct 11, 2026
15 checks passed
@shengyfu
Shengyu Fu (shengyfu) deleted the shengyfu-fix-multiline-match-counting branch October 11, 2026 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

-U --count-matches counts one cross-line match once per covered line and disagrees with --stats

2 participants