Bound and parallelize managed worktree reconciliation - #204

Measured initial-attachment result

At immutable RC2 source 58fdda83495ad9480ebafd3dad44d181039b5023, a fresh warm-base attachment completed in 186.144162 seconds, versus 542.469240 seconds for a same-binary standalone full build: 2.914x faster / 65.69% less wall time in the measured Windows/ReFS pair. Full reconciliation read 293,288 files, consuming 6,894,175,823 input bytes under the unchanged file-size/probe limits, with zero content reads avoided. The searchable file-size cap remains 64 MiB; the managed read/probe limit is 64 MiB + 1 byte. The read counter does not prove every physical file was consumed to EOF.

Those measurements are not from the current PR head. The guard-order correction, user-directed recurring-mode change and subsequent follow-ups below were made afterward; no new private-corpus benchmark was run.

Changes

• Capture effective read limits and policy/allocation versions in WorkPermit. Replace per-file catalog reads with mutation-aware cached control checks, retaining bounded fallback refresh for independent catalog changes and native owner-guard closure, plus forced publication checks.
• Reuse the normal builder's file-count/byte batch planning, but keep strict handle-rooted reads. Share compute, input-byte and parent-handle admission across the namespace rather than creating a pool per attachment.
• Share pinned parent readers among siblings, reserve revalidation headroom before opening handles, and release native handles at batch boundaries. Bound unmanaged/v1 parent retention too; Unix retains ancestor-ID observations without retaining every ancestor descriptor. Those observations are mutation-detection inputs, not lifetime pins or continuity proof.
• Join/drain workers on failure. Retry only recoverable parallel private-memory pressure once at serial width under the same limits; cancellation, policy, identity, deadline and I/O failures remain fatal. One assembler preserves rooted-read, retained-root, input-epoch and policy/allocation checks; final ancestor-ID comparisons add conservative mutation detection, not an atomic filesystem snapshot.
• Expose phase timings, overlapping worker/catalog diagnostics and admitted-capacity peaks. Document the remaining orchestration and receipt/readiness accounting gaps explicitly.
• Remove timer-driven safety repair for healthy native managed views, as explicitly requested. Preserve chosen/fallback polling and all observed invalidation and explicit repair paths; report configured and effective watch modes separately.

Measured acceptance and caveats

The warm-generation receipt proves reused_generation=true, published=false and blobs_read=0; worktree verification reused 287,319 base files and extracted one file. All 293,287 listed paths agreed. Public positive and absent probes agreed across local-index, managed and --no-index searches. Owned helpers shut down cleanly and the original checkout was preserved.

This is one ordered pair, standalone first, with no file-cache flush. Implementation-heavy work was paused, but the host was not exclusive. Native watching was disabled to isolate initial attachment. The measured root had never been attached; its compatible persisted base was originally produced by RC1 (1.2.0), while both measured operations used the same immutable RC2 1.2.1 executable. RC1's failed timing remains a separate diagnostic, not this comparison's baseline.

The namespace still had one operation worker / one allocation work slot, with unchanged policy/allocation capacities. Allocation version 3 is not a limit increase. Worker/catalog diagnostics overlap wall phases; resource peaks are admitted accounting, not sampled RSS, CPU or descriptor usage. The input peak uses a conservative six-times read/decode charge plus 65,536 bytes of scratch for one serialized oversized input, not six allocated buffers.

Full timings, counters, capacities, executable/harness hashes and caveats are in the measured benchmark documentation. Only the sanitized aggregate report was used for publication.

Post-measurement follow-ups

Commit 5d58e312ca40b03f669079c46d0d12883ee479f6 checks growth past the opened length before the managed blob limit. A file opened at exactly blob_bytes that then grows by one byte now returns UnstableFile; a stable initially oversized file still returns checkout-blob-read-limit. Deterministic regressions cover both boundaries. Limits and publication checks are unchanged.

Commit 187a0dbd8de4e460fd227bbec39fffb463053c9f implements the user-directed change from the previously retained native safety pass. It uses each monitor's effective mode for scheduling, adds event-driven-native-reconciliation capability and mode-aware diagnostics, and updates help/docs/agent guidance. Namespace/maintenance status distinguishes configured mode from current native/polling/disabled monitor counts, including mixed native/fallback namespaces.

Commit f0cd8209d828d4eab9196b20feecabb423184501 makes the test timer independent of the platform's Instant epoch. It injects elapsed durations through the real scheduler instead of backdating timestamps, including the retry timer. The production polling/retry predicates are unchanged. Native assertions at 121 seconds and one day remain; fallback polling also checks 119,999 ms, exactly 120 seconds, 121 seconds and one day without sleeping to cross those boundaries.

Commit 27910e3a26cf01a3160a99ee027611bce6336bb5 renames ParentEvidence::verify to ParentObservation::check_current_ids and clarifies its scope in code, README and the managed contract. It adds a managed/unmanaged regression for ordinary post-batch ancestor replacement rejecting publication and successful full repair, plus a point-in-time negative control. This is a naming/documentation/regression correction, not a runtime behavior change or elimination of the ID-reuse/ABA window.

Rooted-read contract and observation limit

Initial attachment and restore still perform the required rooted reads. Original and reopened child handles stay alive across read verification; batch-end comparisons run while scoped batch guards remain held. Retained-root and input-epoch checks independently gate publication. The bounded parent/FD admission and its revalidation headroom are unchanged; matching observed ancestor IDs do not replace required content reads.

An ancestor-ID comparison is not a lifetime proof. Filesystems may reuse IDs after handles close, and paths may change after the final comparison itself. The baseline already documented per-read verification rather than an atomic filesystem snapshot and did not retain every ancestor through publication. A synthetic baseline/current comparison confirmed that baseline accepts an ordinary parent replacement after verified reads while the new observation check rejects it; both can race a later child-file replacement. That probe did not reproduce native ID reuse. This follow-up preserves the original non-snapshot contract; it does not add uninterrupted ancestor continuity or claim that ABA is excluded. Retaining the whole tree's handles would violate the bounded-FD design.

This pre-existing non-snapshot limit is separate from the later decision about missed-notification staleness. Disabling the native safety timer is not the justification for the rooted-read contract and does not relax initial verified reads.

Accepted freshness tradeoff

Silently missed changes can leave healthy native views stale indefinitely, even while ready. This is accepted staleness, not proof of change coverage. No clean-Git shortcut or journal-coverage claim is introduced. Strict initial-attachment/restore verification, secure reads, publication checks, watcher-driven invalidation, explicit full refresh, and reported rescan/error recovery remain intact. Legacy v1 daemon and ordinary/unmanaged refresh policies are unchanged. Use explicit refresh or --no-index when watcher-dependent freshness is insufficient.

The measured RC2 executable retained the old native safety pass; its benchmark used --no-watch. The later recurring-mode behavior therefore differs, and the published timing does not establish an idle-maintenance saving.

Qualification

• Measured-source CI run 38500502076: all nine jobs passed at 58fdda83495ad9480ebafd3dad44d181039b5023, including Linux/macOS/Windows and Windows Rust 1.88 release qualification.
• Original focused core/lifecycle/daemon regressions covered single-worker execution, policy/cancellation/owner invalidation, drained memory-pressure retry, restored-mtime repair, and secure ancestor/child checks. Linux/macOS CI also passed the isolated low-FD managed and unmanaged regression.
• For the guard-order fix, the exact-boundary growth test failed before the fix and passed afterward; the stable oversized case retained its blob-limit error. All five targeted read/worker-drain/private-budget regressions, formatting and core library/test Clippy passed. CI for that follow-up also passed.
• For event-driven native mode, the deterministic old-interval test reproduced the old predicate's unwanted epoch advance. Nine focused integration tests passed, covering timer-only no-op at 121 seconds and one day, real watcher edits, explicit full refresh, chosen polling, native-budget/error fallback, mixed-view diagnostics, --no-watch, rescan/sparse parity, publication recovery, and unchanged legacy behavior. Four CLI help/default/boundary/conflict tests and the core checkpoint-restore verification regression passed. All-target CLI Clippy with hooks, hook-free Rust 1.88 check, formatting and diff checks passed. That change's CI run tracks 187a0dbd8de4e460fd227bbec39fffb463053c9f.
• For timer-hook portability, four focused integration tests passed, covering healthy native, failed-native polling, explicitly chosen/budget-fallback polling and --no-watch. All-target CLI Clippy with hooks, hook-free Rust 1.88 check, formatting and diff checks passed. CI run 38518099233 passed all nine jobs at f0cd8209d828d4eab9196b20feecabb423184501.
• For ancestor observations, 18 focused Windows core tests passed, covering rooted reads, the managed/unmanaged post-batch replacement gate and full repair, point-in-time observation limits, input epochs, error handling, cancellation, drained memory-pressure retry and parent admission. Core/CLI all-target Clippy and Rust 1.88 checks with hooks, formatting and diff checks passed. Both new ancestor regressions and the unchanged Unix-only isolated low-FD regression also passed in this head's Ubuntu job before its separate lifecycle failures; the low-FD result is not a local Windows result.
• Current-head CI run 38522460770 is green after one failed-jobs retry (attempt 2) at 27910e3a26cf01a3160a99ee027611bce6336bb5. The first attempt passed seven jobs and failed in lifecycle tests unchanged from the preceding green head: two immediate-lock assertions on Ubuntu and a collection-budget assertion in Windows Rust 1.88. Only those two failed jobs were rerun once, without source changes; both passed and the seven original successes were retained. First-attempt failure evidence is preserved separately from the focused ancestor results.

Based on main's existing 1.2.1 version; no independent version bump, release or tag.