Skip to content

fix(tui): register reset completion-pause timer before publishing step 3#1573

Merged
Aaronontheweb merged 4 commits into
netclaw-dev:devfrom
Aaronontheweb:fix/init-reset-completion-pause-race
Jul 3, 2026
Merged

fix(tui): register reset completion-pause timer before publishing step 3#1573
Aaronontheweb merged 4 commits into
netclaw-dev:devfrom
Aaronontheweb:fix/init-reset-completion-pause-race

Conversation

@Aaronontheweb

Copy link
Copy Markdown
Collaborator

Summary

Fixes a lost-wakeup race in the "existing install" reset flow that intermittently hangs Netclaw.Cli.Tests in CI (300s blame-hang → dump). The fix is a signal-ordering change: register the completion-pause timer before publishing the step-3 "Purge complete" signal, so the signal never precedes the resource it vouches for.

This was found via hang-dump forensics on an unrelated CI failure and is independent of the PR that surfaced it (#1568) — it fixes a pre-existing race, not that PR's changes.

Root cause — signal before resource

RunResetAsync runs off the UI loop. Its old shape:

  1. PublishOnLoop(... CurrentProgressStep = 3 ...) — publishes "Purge complete"
  2. await Task.Delay(CompletionPause, _timeProvider, ct)registers the completion-pause timer, then awaits it

Observers (including the test helper) gate on step 3 to know deletion finished. But step 3 was published in step (1), before the timer that (2) creates even exists. On a virtualized clock, a clock Advance that lands in the window between "step 3 published" and "timer registered" is dropped — a classic lost wakeup. The delay never fires and the reset hangs forever awaiting a wakeup that will never arrive.

Dump evidence (CI run 28676578217, PR #1568)

Netclaw.Cli.Tests blame-hung after 300s. Dump forensics:

  • Test InitExistingInstallViewModelTests.FullReset_AfterBothConfirmations_DeletesEverything was parked at await vm.ResetTask!.WaitAsync(ct).
  • Its CompleteResetAsync helper had spent every virtual-clock advance — loop counter i == 10 (all 10 advances exhausted).
  • FakeTimeProvider was left holding exactly one registered waiter — i.e. RunResetAsync registered its Task.Delay(CompletionPause, ...) timer after the final Advance. The advance-before-registration window is precisely the lost wakeup.

The fix — happens-before, deterministic

Create the completion-pause timer first, then publish step 3, then await the captured task:

var completionPause = Task.Delay(CompletionPause, _timeProvider, ct);

PublishOnLoop(() =>
{
    CurrentProgressStep.Value = 3;   // "Purge complete" — now the timer already exists
    ...
}, ct);

try
{
    await completionPause;
}
...

Determinism argument: creating the timer first makes "step 3 observed" a happens-before guarantee that the pause timer already exists. So a single clock advance always fires it — there is no window in which an advance can be lost. On the real clock this shifts timer registration a few microseconds earlier; the visible "Purge complete" dwell is unchanged.

The shared test helper CompleteResetAsync is simplified to match the new invariant — wait for step 3, then do exactly one advance:

await WaitForProgressAsync(vm, 3);
_time.Advance(InitExistingInstallViewModel.CompletionPause);
await vm.ResetTask!.WaitAsync(TestContext.Current.CancellationToken);

The old for (i < 10) { Advance; yield } retry loop only ever masked the advance-before-registration lost wakeup on loaded CI runners; it is deleted, not retuned.

Pre-existing status

This race is pre-existing. Prior commits band-aided the symptom with advance/yield retry loops rather than fixing the ordering:

This PR removes the race at the source, so the retry loop is no longer needed.

Coverage

All three sibling tests exercise the fixed code through the shared CompleteResetAsync path:

  • FullReset_AfterBothConfirmations_DeletesEverything
  • SetupOnlyReset_DeletesConfigButKeepsMemoryAndSessions
  • DaemonStopFailureResult_ShowsStatusAndContinuesReset

InitExistingInstallPageTests.cs was analyzed and left untouched — it is immune (it completes via the cancellation path, not the completion-pause timer).

Validation

  • Build: dotnet build src/Netclaw.Cli.Tests/Netclaw.Cli.Tests.csproj -c Release — 0 warnings, 0 errors.
  • Targeted stress: affected class run 20× in Release (--filter FullyQualifiedName~InitExistingInstallViewModelTests) — 20/20 passed, none hung (15 tests each, ~100 ms/run).
  • Full suite: dotnet test src/Netclaw.Cli.Tests -c Release --no-build1196 passed, 0 failed, 0 skipped.
  • TUI smoke gate: ./scripts/smoke/run-smoke.sh init-wizardOK (all smoke checks passed; doctor exit 2 is expected WARN-only for the temp-dir session base and absent secrets.json in the smoke sandbox).
  • Slopwatch: dotnet slopwatch analyze — 0 issues found.
  • Headers: scripts/Add-FileHeaders.ps1 -Verify — all files have headers.

RunResetAsync publishes CurrentProgressStep = 3 ("Purge complete") and then
evaluates the Task.Delay(CompletionPause, _timeProvider, ct) that registers the
completion-pause timer. Because RunResetAsync runs off the loop, step 3 is the
signal observers gate on to know deletion finished — but it fired BEFORE the
timer it vouches for existed. On a virtualized clock (FakeTimeProvider in tests,
and any consumer advancing the injected TimeProvider) this is a lost wakeup: an
Advance that lands between "step 3 published" and "timer registered" is dropped,
the delay never fires, and the reset hangs forever awaiting a wakeup that will
never come.

Fix: create the completion-pause timer before publishing step 3, then await the
captured task. This makes "step 3 observed" a happens-before guarantee that the
pause timer already exists, so a single clock advance always fires it
deterministically. On the real clock this shifts timer registration a few
microseconds earlier; the visible "Purge complete" dwell is unchanged.

Evidence: hang-dump forensics of CI run 28676578217 (PR netclaw-dev#1568, Netclaw.Cli.Tests
blame-hang after 300s). Test FullReset_AfterBothConfirmations_DeletesEverything
was parked at `await vm.ResetTask!.WaitAsync(ct)`; its CompleteResetAsync helper
had exhausted its advance loop (counter i == 10, all virtual-clock advances
spent) and FakeTimeProvider was left holding exactly one orphaned waiter — the
delay registered after the final Advance. Prior commits 850cf8c (netclaw-dev#1525) and
eb01b49 (netclaw-dev#1536) band-aided the symptom with advance/yield retry loops; this
removes the race at the source.

The test helper CompleteResetAsync is shared by all three sibling tests
(FullReset_AfterBothConfirmations_DeletesEverything,
SetupOnlyReset_DeletesConfigButKeepsMemoryAndSessions,
DaemonStopFailureResult_ShowsStatusAndContinuesReset); it now waits for step 3
then does exactly one advance, deleting the retry loop that only ever masked the
lost wakeup on loaded runners.
@Aaronontheweb Aaronontheweb added tui Terminal UI (Termina) issues bug Something isn't working labels Jul 3, 2026

@Aaronontheweb Aaronontheweb left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no real behavior changes, just a test bug fix. Hang dump analysis confirmed this.

@Aaronontheweb
Aaronontheweb enabled auto-merge (squash) July 3, 2026 19:51
@Aaronontheweb
Aaronontheweb merged commit 134abe1 into netclaw-dev:dev Jul 3, 2026
15 checks passed
Aaronontheweb added a commit that referenced this pull request Jul 3, 2026
The mcp-permissions-server-list frame intermittently captured the raw
shell transcript instead of the rendered `netclaw mcp permissions` TUI
(CI run 28680686756, job "Screenshot Regression (Linux)" on PR #1573).

Root cause: both Wait+Screen anchors preceding the Frame 1 Screenshot
matched text that is already sitting in the shell scrollback BEFORE the
TUI paints, so both waits returned instantly:

- `Wait+Screen@15s /smoke-math/` matched the setup output "Added MCP
  server 'smoke-math' (stdio)" and "...adjust approvals for
  'smoke-math'." printed by `netclaw mcp add`.
- `Wait+Screen@5s /3 tools/` matched the tape's OWN typed readiness
  loop, still visible on screen: `until netclaw mcp list 2>/dev/null |
  grep -q 'connected (3 tools)'; do sleep 2; done`.

With both anchors pre-satisfied, the only remaining delay was the
documented `Sleep 3s` settle guard. On a loaded CI runner the TUI took
longer than 3s to paint, so Screenshot fired while the screen still
showed the shell transcript, failing the byte-for-byte baseline
comparison. This violates the tapes/README.md rule to anchor every
step on stable substrings from the NEXT view.

Fix: anchor Frame 1 on strings that only the rendered TUI produces:

- `MCP Permissions` — the page title from
  McpToolPermissionsPage.BuildHeader; proves the alternate screen
  buffer painted. Nothing in the setup transcript prints this string.
- `Connected, 3 tools` — the exact server-row text from
  McpToolPermissionsPage.BuildServerList, rendered as
  "{Name}  ({Status}, {ToolCount} tools)". The daemon reports the
  state as PascalCase "Connected" (McpConnectionState.ToString());
  the CLI transcript form is lowercase and comma-less
  ("connected (3 tools)"), so the two can never collide.

The `Sleep 3s` settle guard is kept: it protects against a different
hazard (daemon state refresh immediately after first render) and the
committed baseline was captured with it in place.

The tool-grid frame's anchors (/record-tasks/, /Server default:/) and
the other screenshot tapes were audited for the same hazard: all of
their pre-capture anchors match only post-TUI content, so no other
changes were needed.

Validation: `scripts/smoke/run-smoke.sh screenshots` passed 3
consecutive runs locally, all 9 frames matching baselines
(mcp-permissions-server-list at AE=0 each run).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working tui Terminal UI (Termina) issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant