Skip to content

Commit 187a0db

Browse files
shengyfuCopilot
andcommitted
Remove periodic safety repair for native managed views
Honor the requested event-driven staleness tradeoff for healthy native managed views. Preserve chosen and fallback polling, strict initial/restore verification, explicit refresh and error recovery. Report configured and effective update modes separately, cover the former timer boundary deterministically, and retain the original benchmark attribution. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 9776b17f-6a2d-4c47-9e89-0f8113d14b95
1 parent 5d58e31 commit 187a0db

8 files changed

Lines changed: 680 additions & 171 deletions

File tree

‎AGENTS.md‎

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -98,10 +98,17 @@ Legacy/unmanaged v1 storage remains retain-all.
9898
Managed attachments use bounded parallel verified-read batches without raising
9999
namespace operation-worker limits. Inspect `last_reconcile.timings` and its
100100
read/reuse/resource counters to distinguish queueing, discovery, verification,
101-
catalog/control and checkpoint work. This does not eliminate periodic repair:
102-
`reconciliation_policy` reports the full-verification fallback when trustworthy
103-
change coverage is unavailable. A clean Git status is never proof of indexed
104-
checkout bytes, even for another worktree at the same commit.
101+
catalog/control and checkpoint work. Healthy native managed views have no
102+
periodic safety rescan: silently missed notifications can leave results stale
103+
indefinitely, even while ready. This is an accepted staleness tradeoff, not
104+
proof of change coverage. `views.refresh` explicitly verifies all admitted
105+
contents; use `--no-index` when current disk bytes are required.
106+
`--watch-mode poll` and actual native-watcher failure/budget fallback still
107+
perform full verification after `--poll-interval` seconds (default 120).
108+
`reconciliation_policy` distinguishes configured mode from each view's effective
109+
polling and reports namespace monitor counts. First attachment/restore and
110+
reported watcher errors/rescans retain full verification. A clean Git status
111+
never replaces those reads, even for another worktree at the same commit.
105112

106113
Moving an existing v1 integration to managed v2 requires this client lifecycle,
107114
not just a startup flag. Detach all v1 leases and drain in-flight operations

‎README.md‎

Lines changed: 24 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -266,7 +266,7 @@ Automatic refresh has two modes, configured on `tgrep serve`:
266266
| Flag | Default | Effect |
267267
|------|---------|--------|
268268
| `--watch-mode <auto\|poll>` | `auto` | Prefer native notifications with polling fallback, or use polling only |
269-
| `--poll-interval <SECONDS>` | `120` | Wait after each polling reconciliation completes; range 1-86400 |
269+
| `--poll-interval <SECONDS>` | `120` | Wait after each polling reconciliation completes; range 1-86400; does not schedule healthy native managed views |
270270
| `--watch-budget <N>` | `8192` | Conservative process-local native watch ceiling; range 1-4294967295 |
271271
| `--no-watch` | off | Disable all automatic refresh: native watching, polling, and periodic reconciliation |
272272

@@ -286,7 +286,8 @@ finishes; scan time and ongoing builds/saves add to the delay. Queries do not
286286
defer polling. Queue overflows and native rescan notifications also request
287287
reconciliation; read-only access events are discarded.
288288

289-
Native mode reconciles about once an hour to recover missed notifications.
289+
Ordinary (non-shared) native mode reconciles about once an hour to recover missed
290+
notifications.
290291
It waits for a two-minute gap in queries, deferring no longer than four hours.
291292
`--poll-interval` does not change this native safety cadence.
292293

@@ -856,8 +857,8 @@ options, `--no-ignore` variants, `--text`, `--binary`, `--follow`,
856857
`--one-file-system`, `--ignore-file`, passthru, files-without-match and zero-count
857858
output scan with a diagnostic rather than claiming incompatible coverage.
858859

859-
**Freshness:** native notifications close readiness as they arrive; bounded
860-
hints are an optimization, not proof that all events arrived. Every automatic
860+
**Legacy v1 freshness:** native notifications close readiness as they arrive;
861+
bounded hints are an optimization, not proof that all events arrived. Every automatic
861862
mode also fully reconciles after `--poll-interval` seconds (default 120, measured
862863
after completion). Native registration failures/budget exhaustion fall back to
863864
polling. `--watch-mode poll` skips native registration; `--no-watch` disables both
@@ -960,10 +961,21 @@ A [same-binary Windows/ReFS measurement](SHARED_INDEX_BENCHMARKS.md#large-checko
960961
completed a fresh warm-base attachment in 186.144 seconds versus 542.469 seconds
961962
for a standalone full build (2.914x), while still reading every required file.
962963
That result is one ordered pair without a file-cache flush, not a universal speedup.
963-
This initial-pipeline optimization does **not** eliminate recurring full repair:
964-
without trustworthy change coverage, the existing 120-second fallback remains.
965-
`reconciliation_policy` in status reports that limitation and the configured
966-
interval; neither timestamps nor an unavailable ReFS journal prove coverage.
964+
Healthy native managed views do **not** run the former 120-second periodic
965+
safety rescan. Watcher notifications still invalidate and reconcile views, but
966+
silently missed changes can leave results stale indefinitely, even while ready.
967+
This deliberately accepts staleness, not proven change coverage; timestamps,
968+
clean Git status and an unavailable journal do not prove that nothing changed.
969+
Initial attachment/restore, explicit `views.refresh`, and reported watcher
970+
errors/rescans retain verified repair. Use `--no-index` for current disk bytes.
971+
972+
Explicit `--watch-mode poll` and actual native-watcher failure/budget fallback
973+
still perform full reconciliation after `--poll-interval` seconds (default 120)
974+
following completion. `--no-watch` disables watching and polling, not initial
975+
verification or explicit refresh. `reconciliation_policy` distinguishes these
976+
per-view modes and namespace monitor counts. This recurring-mode change is
977+
subsequent to the immutable benchmark source, not a measured idle-work saving.
978+
Legacy v1 and ordinary-server refresh policies are unchanged.
967979

968980
`views.advance` prepares and reconciles a complete replacement while unrelated
969981
ready views continue serving. The current exact commit, physical generation,
@@ -1153,8 +1165,10 @@ publishing buffered results, reenter `with_snapshot` and reject a changed epoch.
11531165
Later reads through that handle remain caller-owned: report read errors and call
11541166
`invalidate_all()` after leaving the guard rather than publishing partial results.
11551167
Do not reenter the view from its closure. A refresh acknowledges processed hints,
1156-
**not an atomic filesystem snapshot**. Periodic full reconciliation remains necessary; no-watch
1157-
callers must explicitly refresh, and final reads can race subsequent edits.
1168+
**not an atomic filesystem snapshot**. For verification independent of
1169+
notifications, use explicit or polling full repair: healthy native managed views
1170+
no longer promise eventual periodic repair, and no-watch callers must explicitly
1171+
refresh. Final reads can race subsequent edits.
11581172

11591173
Full verification reads, auto-decodes and hashes checkout bytes; clean Git
11601174
status, blob identity and index stat data are never byte-equivalence proofs.

‎SHARED_INDEX_BENCHMARKS.md‎

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -153,11 +153,21 @@ limit, not a higher searchable file-size cap or a count of allocated buffers.
153153
It does not enlarge the 64 MiB concurrent-input gate.
154154
Private charges remained within the unchanged 8 GiB private-work limit.
155155

156-
**Recurring full-reconciliation elimination is not implemented.** The normal
157-
120-second full-repair fallback remains when trusted change coverage is absent;
158-
clean Git status never substitutes for verified checkout bytes. The benchmark's
159-
disabled watcher isolates initial attachment and does not demonstrate an idle
160-
maintenance improvement or authorize changing the default repair interval.
156+
**Post-measurement recurring-mode change:** measured source `58fdda8` retained
157+
the normal 120-second native safety pass. A subsequent user-directed change
158+
removes that timer for healthy native managed views, accepting that silently
159+
missed notifications may leave results stale indefinitely rather than claiming
160+
trusted change coverage. Explicit polling and actual native-watcher-unavailable
161+
fallback still perform full verification after `--poll-interval` (default 120
162+
seconds); initial/restore verification, explicit refresh and reported
163+
watcher-error/rescan recovery remain intact. Legacy v1 and ordinary-server
164+
policies are unchanged.
165+
166+
That later behavior is **not part of the measured executable**. The benchmark's
167+
disabled watcher isolates initial attachment and does not measure idle
168+
maintenance savings. Timings remain attributed to immutable `58fdda8`, not
169+
subsequent PR heads; clean Git status never substitutes for initial verified
170+
checkout reads.
161171

162172
## Managed lifecycle measurements
163173

‎SHARED_WORKTREE_INDEXES.md‎

Lines changed: 44 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -654,7 +654,7 @@ refresh and checkpoint publication finish, including failed checkpoint retries.
654654
Candidate files are opened through the view's retained snapshot reader. Every
655655
query ends with a root-verified epoch check, including empty and filename-only
656656
responses; replacing the directory cannot pair old candidates with a new root.
657-
Both auto and poll modes perform full
657+
In the legacy v1 daemon, both auto and poll modes perform full
658658
verification every `--poll-interval` seconds after completion (default 120), also
659659
repairing missed bytes, ignores and external Git/global configuration changes.
660660
Native notifications allow earlier incremental repair. `--no-watch` disables
@@ -1299,26 +1299,52 @@ Under the view's work/publication gates, tgrep checks the exact current pin,
12991299
checkpoint and input epoch before completing such a job without changing the
13001300
ready view or invalidating its readers. Its original token/request is retained;
13011301
the terminal receipt reports `coalesced: true`, the existing reconciled epoch
1302-
and zero new reconciliation work. This does not postpone periodic verification.
1303-
Explicit refresh always verifies; periodic repair, watcher uncertainty and
1302+
and zero new reconciliation work. This does not postpone polling verification.
1303+
Explicit refresh always verifies; polling repair, watcher uncertainty and
13041304
rescan events invalidate for full verification before queuing reconciliation.
13051305
A newer invalidation therefore cannot be skipped by the coalescing check.
13061306

1307-
**Recurring-repair limitation:** the bounded initial pipeline does not eliminate
1308-
periodic full verification. This implementation has no integrated, lossless
1309-
checkout change journal. File timestamps, Windows `ChangeTime`, a clean Git
1310-
status, or a drained watcher queue do not prove absence of unreported writes
1311-
(including active/mapped writers). An unavailable/inactive journal, including on
1312-
ReFS, must not be treated as an empty journal. First attachment/restore, explicit
1313-
full refresh, watcher error/overflow, Git/ignore changes and periodic fallback
1314-
therefore continue to require full verification. The default 120-second interval
1315-
after completion is retained, not silently lengthened or disabled.
1316-
Online `namespace.status`, `views.status` and ordinary `status` expose
1317-
`reconciliation_policy` with `periodic_full_verification`, `interval_ms`,
1318-
`incremental_coverage: "unavailable"`, `change_journal: "not-integrated"`,
1319-
`full_verification_reason: "no-trusted-change-coverage"` and repair triggers.
1320-
`--no-watch` explicitly disables automatic repair; it does not supply coverage
1321-
evidence or skip first-attachment verification.
1307+
**Managed update policy:** healthy native-watcher views have no timer-driven
1308+
full-reconciliation safety pass. Elapsing the former 120-second interval does
1309+
not invalidate or reread an otherwise quiet ready view. Silently missed changes
1310+
can therefore remain stale indefinitely, even while ready. This is an explicitly
1311+
accepted staleness tradeoff, not proven coverage. There is no integrated lossless
1312+
checkout change journal; timestamps, Windows `ChangeTime`, clean Git status and
1313+
a drained watcher queue still do not prove absence of unreported writes
1314+
(including active/mapped writers). An unavailable/inactive journal is not an
1315+
empty journal and does not justify skipping initial content verification.
1316+
1317+
First attachment/restore, explicit full refresh, reported watcher error/overflow
1318+
or rescan, and observed Git/ignore changes retain full verification and the
1319+
existing publication gates. Normal native file events still invalidate their
1320+
paths. Explicit `--watch-mode poll` and actual native-watcher unavailability,
1321+
failure or budget exhaustion use periodic full reconciliation as their update
1322+
mechanism, not a redundant native safety pass. They wait `--poll-interval`
1323+
seconds after completion (default 120). `--no-watch` disables both automatic
1324+
update mechanisms but not initial verification or explicit refresh. Legacy v1
1325+
and ordinary-server policies are unchanged.
1326+
1327+
`hello` advertises `event-driven-native-reconciliation`. Online `views.status`
1328+
and ordinary `status` expose the effective `watch_mode`, `watch_fallback` and
1329+
`reconciliation_policy`. It reports `native_periodic_full_verification: false`,
1330+
`native_missed_changes: "may-remain-stale-indefinitely"`,
1331+
`incremental_coverage: "unavailable"` and `change_journal: "not-integrated"`.
1332+
`periodic_full_verification` is true only for an effectively polling view;
1333+
`interval_ms` is then its active interval and `full_verification_reason` is
1334+
`"polling-update-mechanism"`. Both fields are null otherwise. The configured
1335+
`poll_interval_ms` remains visible even when no periodic work is enabled;
1336+
`configured_watch_mode` is `auto`, `poll` or `disabled`.
1337+
1338+
Online `namespace.status` and `maintenance.status` distinguish configuration
1339+
from effective per-view behavior: the same policy has
1340+
`scope: "monitored-views"` plus `native_views`, `polling_views` and `disabled_views`
1341+
counts. Its `periodic_full_verification` means at least one monitored view is
1342+
polling, not that all views have a safety timer. A namespace configured as
1343+
`auto` can contain both healthy native views and fallback-polling views. With
1344+
no polling monitors its active interval/reason are null, even if polling is
1345+
configured for future views. Repair triggers include `poll-interval` only when
1346+
polling is active. Use explicit refresh or `--no-index` when watcher-dependent
1347+
freshness is insufficient.
13221348

13231349
`views.adaptive` takes `view`, `owner`, `expected_version`, `allocation_version`
13241350
in an operation envelope. Adaptive policy evaluates the worktree's captured

‎tgrep-cli/src/main.rs‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -694,6 +694,7 @@ enum Command {
694694
watch_mode: serve::WatchMode,
695695

696696
/// Seconds to wait after a polling reconciliation completes (1-86400).
697+
/// Managed v2 uses this only for polling views, not healthy native watchers.
697698
#[arg(
698699
long,
699700
default_value_t = 120,
@@ -1896,6 +1897,23 @@ mod tests {
18961897
}
18971898
}
18981899

1900+
#[test]
1901+
fn managed_poll_interval_help_distinguishes_native_watchers() {
1902+
let help = parse(&["serve", "--help"]).err().expect("serve help");
1903+
assert_eq!(help.kind(), clap::error::ErrorKind::DisplayHelp);
1904+
let help = help.to_string();
1905+
assert!(
1906+
help.contains("Seconds to wait after a polling reconciliation completes"),
1907+
"{help}"
1908+
);
1909+
assert!(
1910+
help.contains(
1911+
"Managed v2 uses this only for polling views, not healthy native watchers"
1912+
),
1913+
"{help}"
1914+
);
1915+
}
1916+
18991917
#[test]
19001918
fn refresh_defaults_and_explicit_modes() {
19011919
for (args, expected_mode, expected_no_watch) in [

0 commit comments

Comments
 (0)