Repository navigation
fix(mcp): report freshness with the indexer's own mtime source (#1714) - #1787
Conversation
|
Thanks for opening this — it has been seen, and it is queued. This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence. Current review status: working through a backlog. What that means for this PR, concretely:
Things that will genuinely speed it up whenever review does happen:
If this fixes a bug, a reproduction we can run is worth more than a description of the symptom. Thanks for contributing, and sorry in advance for the wait. |
check_index_coverage freshness recomputed mtime_ns from struct stat, which on Windows truncates to seconds, while the indexer records cbm_path_info_utf8's FILETIME-derived nanosecond value. A byte-identical file therefore never matched and every path was reported metadata_changed, so the freshness field carried no signal on Windows. The pipeline incremental change-detection had the same truncation in its hash comparison. Both readers now compare against the same source the writer uses. Fixes DeusData#1714 Signed-off-by: Ulises Millan Guerrero <[email protected]>
07abd9b to
455e4fb
Compare
|
Thank you for tracing the Windows freshness mismatch to two different timestamp sources. I checked current I have labeled this as a high-priority Windows parsing and correctness bug and routed it for review. The current CI surface is green. Our review queue is full, so detailed review may take a little time. Thank you for converging the readers on the writer's existing source of truth. |
|
Independent Windows confirmation from
The PR Windows CI and regression are green. I could not independently rebuild the PR binary locally because this machine does not have the required MSYS2 CLANG64 toolchain, so this is an independent reproduction plus source-level validation, not a separate binary test. |
|
Approved on merit, and the impact is larger than your description claims — I want to spell that out, because it strengthens the case rather than weakening it. VerifiedBoth truncations are real and both are Windows-only:
macOS uses And the repo already states the rule you are enforcing.
So this is not a new convention — it is an existing, documented invariant that two readers were quietly violating. That is the cleanest possible justification for the change, and I would put that citation in the PR description. The part you understatedYou describe the if (stat_mtime_ns(&st) != h->mtime_ns || st.st_size != h->size) {
changed[i] = true;With a seconds-truncated left side and a FILETIME-derived right side, those can only agree when the sub-second component is exactly zero. So on Windows incremental reindex has been degenerating into a full reindex on essentially every run — that is a performance defect on top of a correctness one, and on a large repository it is the more expensive of the two. The Before mergePlease tick or explain the lint box. You are 205 commits behind Nothing to change in the code. Thank you — this is a well-isolated fix to a bug that made a correctness signal actively harmful on one platform. |
|
Merged — with an apology attached. This was approved on merit on 2026-09-02 and then sat green for eighteen days. Nothing was outstanding from you; the delay was entirely ours. Reading it back before merging, the maintainer note from 09-02 holds up: the impact is larger than your own description claimed. I also want to credit @chao-hsu-14 here: their independent confirmation on 0.10.8 — Thank you both. Sorry it took so long to act on work that was ready. |
DeusData#1787 landed correct production code and a test that cannot pass on today's main. The test is the only thing wrong, and only in how it asks. check_index_coverage answers in the compact table by default, where a field name never appears: main.go main.go coverage_unavailable metadata_match read_source_and_reindex [] The test asserts on the JSON field spelling: strstr(response, "\"freshness\":\"metadata_match\"") Both halves are right; they were simply written against different response formats. The branch was cut on 2026-09-02, before that default changed, and was never rebased -- its CI green is from a main that no longer exists. The neighbouring coverage tests already pass "format":"json" for exactly this reason; this one predates them. So the request gains format=json and the assertions are untouched. Asserting the field rather than a bare strstr for "metadata_match" is what keeps the check exact: the value also appears in the table's recommended_action column and would be matched by any other path's row. Verified: build/c/test-runner mcp -> 318 passed, 0 failed, 4 skipped. Before this change the same suite reported 317 passed, 1 failed on identical production code. What this is NOT: a production defect. coverage_path_freshness on main uses cbm_path_info_utf8, exactly as DeusData#1787 intended, and the Windows second-vs- nanosecond truncation it fixes is gone. I checked that by reading origin/main rather than the local checkout, which was stale at 59a05eb and still carried the old helper -- that misreading briefly convinced me the merge had dropped the fix, and it had not. Signed-off-by: Martin Vogel <[email protected]>
DeusData#1787 landed correct production code and a test that cannot pass on today's main. The test is the only thing wrong, and only in how it asks. check_index_coverage answers in the compact table by default, where a field name never appears: main.go main.go coverage_unavailable metadata_match read_source_and_reindex [] The test asserts on the JSON field spelling: strstr(response, "\"freshness\":\"metadata_match\"") Both halves are right; they were simply written against different response formats. The branch was cut on 2026-09-02, before that default changed, and was never rebased -- its CI green is from a main that no longer exists. The neighbouring coverage tests already pass "format":"json" for exactly this reason; this one predates them. So the request gains format=json and the assertions are untouched. Asserting the field rather than a bare strstr for "metadata_match" is what keeps the check exact: the value also appears in the table's recommended_action column and would be matched by any other path's row. Verified: build/c/test-runner mcp -> 318 passed, 0 failed, 4 skipped. Before this change the same suite reported 317 passed, 1 failed on identical production code. What this is NOT: a production defect. coverage_path_freshness on main uses cbm_path_info_utf8, exactly as DeusData#1787 intended, and the Windows second-vs- nanosecond truncation it fixes is gone. I checked that by reading origin/main rather than the local checkout, which was stale at 59a05eb and still carried the old helper -- that misreading briefly convinced me the merge had dropped the fix, and it had not. Signed-off-by: Martin Vogel <[email protected]>
DeusData#1787 landed correct production code and a test that cannot pass on today's main. The test is the only thing wrong, and only in how it asks. check_index_coverage answers in the compact table by default, where a field name never appears: main.go main.go coverage_unavailable metadata_match read_source_and_reindex [] The test asserts on the JSON field spelling: strstr(response, "\"freshness\":\"metadata_match\"") Both halves are right; they were simply written against different response formats. The branch was cut on 2026-09-02, before that default changed, and was never rebased -- its CI green is from a main that no longer exists. The neighbouring coverage tests already pass "format":"json" for exactly this reason; this one predates them. So the request gains format=json and the assertions are untouched. Asserting the field rather than a bare strstr for "metadata_match" is what keeps the check exact: the value also appears in the table's recommended_action column and would be matched by any other path's row. Verified: build/c/test-runner mcp -> 318 passed, 0 failed, 4 skipped. Before this change the same suite reported 317 passed, 1 failed on identical production code. What this is NOT: a production defect. coverage_path_freshness on main uses cbm_path_info_utf8, exactly as DeusData#1787 intended, and the Windows second-vs- nanosecond truncation it fixes is gone. I checked that by reading origin/main rather than the local checkout, which was stale at 59a05eb and still carried the old helper -- that misreading briefly convinced me the merge had dropped the fix, and it had not. Signed-off-by: Martin Vogel <[email protected]>
What does this PR do?
check_index_coveragereportedfreshness: "metadata_changed"for every file on Windows, always — including files checked seconds after a successful full re-index. The freshness reader recomputedmtime_nsfromstruct stat(which on Windows truncates to seconds viast_mtime), while the indexer records the FILETIME-derived nanosecond value fromcbm_path_info_utf8. A byte-identical file therefore never matched, so the freshness field carried no signal and agents following the documented guidance were told to distrust a graph that was actually current.Both readers now compare against the same source the writer uses:
coverage_path_freshness(mcp.c) derives the comparison mtime/size fromcbm_path_info_utf8— the exact function the indexer records from — instead of a platform-dependentstatconversion. The Windows-onlyst_mtimetruncation path is gone.pipeline_incremental.c), where it would mark every file changed on Windows; it now usescbm_path_info_utf8too, on all platforms.Checklist
git commit -s) — required, CI rejects unsigned commits (DCO, see CONTRIBUTING.md)make -f Makefile.cbm test)make -f Makefile.cbm lint-ci)