v1.2.5: stop-if-idle fails already-accepted view operations with namespace-admission-closed even when it decides busy #218

Summary

stop-if-idle closes namespace admission before counting protected work and keeps it closed until it reopens admission after a busy decision. When its activity.lock try-lock succeeds, it also holds that lock exclusively for the rest of the decision.

A view operation accepted before the closure can reach an admission check inside that window. Its job then fails terminally with busy/namespace-admission-closed. ViewManager::execute does not defer that reason, so the receipt becomes failed. This happens even though the same probe counted the operation as protected work and returned stopping:false.

SHARED_WORKTREE_INDEXES.md documents the closure ("stop-if-idle first closes admission, then accounts for … A busy decision reopens admission"). It does not say that a probe ending busy can fail work that was already admitted.

Environment

• tgrep v1.2.5 on Windows. The idle/admission code is unchanged since v1.2.4: tgrep-core/src/managed/maintenance.rs and lifetime.rs are identical between the two tags.
• One managed daemon shared by two Copilot runtime processes. One process supervises the daemon; the other attaches a worktree of the same repository.
• Test-fixture policy: workers 1, work_slots 2, idle_timeout_ms 1000, admission_retry_ms 100, operation_timeout_ms 30000. Production uses a much longer idle timeout, which makes the window rarer but does not remove it.
• CPU saturated by concurrent test processes. Each client RPC took 1.1–3.2 s, compared with 250–370 ms unloaded.

Timeline

Times are client-observed seconds, relative to the start of the run.

Other rounds in the same run returned namespace-maintenance-active (31.9, 40.97) or stopping:false without failing anything.

Code path (v1.2.5)

1. In tgrep-cli/src/serve/shared/managed.rs, the "stop-if-idle" handler sets the process admission gate to false for the whole call. During that time the accept loop answers every new connection with daemon-admission-closed, including receipt polls for accepted operations, and the maintenance tick skips. The handler then runs views.drain_released() and calls Namespace::stop_if_idle_with_token.

2. In tgrep-core/src/managed/maintenance.rs, stop_if_idle_inner commits admission='closed' and counts leases, owners, reservations and operations. It then always attempts try_lock_exclusive(activity.lock), even when the counts already show protected work. It runs discard_idle_verification and only then commits admission='open'.

3. Two checks return busy/namespace-admission-closed during that interval:

   • work::ensure_admission, while the catalog has admission!='open';
   • ActivityGuard::acquire, whose shared lock is refused while the probe holds activity.lock exclusively.

   We have not determined which of the two the peer's job hit.

4. In tgrep-core/src/managed/views.rs, ViewManager::execute defers only view-work-active, generation-already-materializing, view-attachment-preparing, object-retirement-in-progress and catalog-checkpoint-readers-active through defer_operation. Any other error, including namespace-admission-closed, is recorded as the operation's terminal result.

Admission closes before the count runs, so the same probe counts every operation accepted before the closure (operations > 0). The probe therefore cannot commit a stop while such an operation exists, and failing the operation's job gains nothing.

Expected

• A probe that finds protected work does not fail admitted work. Operations accepted before the closure, and their jobs, keep running or wait for the decision. They are never failed with namespace-admission-closed.
• Ideally, a probe that can already see protected work returns stopping:false without closing either gate or taking activity.lock exclusively. Busy daemons would then never expose the window.
• During a genuine closure, new requests still get the documented retryable busy rejection before acceptance. They are never accepted and then failed.
• Only a committed stop ends in-flight work, with a reason distinct from a transient closure.

Impact

A session whose attach fails this way falls back to non-indexed search for its lifetime. A retry with a new token cannot recover when the first attempt consumed most of the client's operation deadline.

The Copilot runtime no longer sends stop-if-idle while its own sessions are live, which removes the trigger above. Three cases remain:

• a supervising process with no live sessions racing another process's attach;
• orphan-maintenance probes against a daemon that is serving other clients;
• any other client that sends stop-if-idle.

Possible fix (suggestion)

1. Busy fast path in stop_if_idle_inner and the CLI handler's process gate. Check the cheap counts first, and when any protected work is visible, record the non-stopping decision without closing admission or locking activity.lock. Returning "not stopping" from a slightly stale positive observation is always safe.
2. Defer instead of failing. In ViewManager::execute, treat a transient namespace-admission-closed as deferrable for operations accepted before the closure, via defer_operation. This covers the remaining window, where the pre-check saw no work but an operation was accepted just before closure: its job is delayed instead of failed. Requeued work must resume once admission reopens, and a committed stop must still end it.
3. Deterministic regression tests with the existing IdleAdmissionClosed fault point and managed-test-hooks:
   • Pause a probe after closure while an accepted attach's job runs. Assert that the probe returns busy, the attach completes, and admission reopens.
   • A race test: an attach is either admitted and completes while the daemon stays, or is rejected before acceptance while the daemon stops.

Evidence, including merged daemon and client timelines and receipts, is retained and can be shared on request.