Skip to content

Commit da2580e

Browse files
wpfleger96npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7
andauthored
fix(desktop): clear stale working badges on agent stop/restart (#2803)
When Desktop stops or restarts a managed-agent pair, the harness is SIGKILLed after 1s and `turn_completed` never lands. `activeAgentTurnsStore` then sees the "all turns silent at once" pattern and delays badge cleanup for up to 3 min (the pause that protects live badges during transient relay-stream gaps). This is unnecessary when Desktop itself issued the kill — there is no relay-gap ambiguity. ## What changed **`activeAgentTurnsStore.ts`** — new `clearActiveTurnsForAgent(pubkey)` - Tombstones every live turn for the agent via `recordTerminal` (blocks in-flight `turn_liveness` frames from resurrecting them via `resurrectTurn`) - Removes the agent's entry from `activeTurnsByAgent` - Preserves `lastProcessed` (watermark) — full-buffer replay after the clear is a no-op - Preserves `clockOffsetByAgent` — still valid, harmless **`managedAgentRuntimeHooks.ts`** — clearing at the successful-stop boundary - New `clearActiveTurnsForAgentOnStop(pubkey, relayUrl?)` — relay-scope gate: only clears when the stopped pair's relay matches the active community (pair-scoped), or when an active community is configured (agent-wide ops) - New `restartManagedAgentPair(pubkey, relayUrl, stop, clear, start)` — dependency-injected stop → clear → start sequence; the `restart` branch of `useManagedAgentRuntimeAction`'s `mutationFn` is a single call into it. The clear fires after a successful stop and before start begins, so the badge is gone even when start fails, a failed stop clears nothing, and no clear can run after the new process is spawned (genuinely-new turns are never wiped) - `useManagedAgentRuntimeAction.onSuccess` clears for `stop` actions, before the query-cache update **`managedAgentControlActions.ts`** — `onStopped` callback on `respawnManagedAgentWithRules`, invoked after the stop promise resolves and before start begins **`welcomeKickoff.ts`** — same `onStopped` boundary on `restartWelcomeTeammate` **Call sites covered (all stop/restart UI paths):** - `useManagedAgentRuntimeAction` — pair-scoped stop (`onSuccess`) and restart (`restartManagedAgentPair` in `mutationFn`); Members-sidebar + settings card - `useMembersSidebarActions.handleRespawnAll` — via `onStopped` - `useMembersSidebarActions.handleStopAll` — direct local-stop branch - `useMembersSidebarActions.handleLifecycleAction` — local-stop fallback branch - `useAgentLifecycleActions.handleAgentPrimaryAction` — Agents-tab stop - `useAgentLifecycleActions.handleAgentRestart` — via `onStopped` - `useManagedAgentActions.handleStop` / `handleBulkStopRunning` — Agents screen - `useAutoRestartPolicy` — inline, between stop and start - `restartWelcomeTeammate` call site — via `onStopped` Provider agents are excluded at each site: they go through `!shutdown` (relay message), not a direct harness kill. ## Tests Twelve behavior tests across three files: - `activeAgentTurnsStore.test.mjs` (6) — clear removes the agent's turns and notifies subscribers, other agents untouched; full-buffer replay after clear is a no-op (watermark preserved); late `turn_liveness` frame with timestamp ≤ clear time does not resurrect (tombstone); new `turn_started` after clear is tracked normally; badge gone when stop succeeds even if start fails; new frame during start-pending does not resurrect the cleared badge - `managedAgentControlActions.test.mjs` (3) — `onStopped` fires on stop-success/start-failure; does not fire on stop-failure; strict stop → `onStopped` → start ordering - `managedAgentRuntimeHooks.test.mjs` (3) — pair-restart seam: clear ran when start fails (rejection propagates); stop failure invokes neither clear nor start; strict stop → clear → start ordering --------- Signed-off-by: Will Pfleger <[email protected]> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
1 parent 4d91195 commit da2580e

11 files changed

Lines changed: 582 additions & 13 deletions

‎desktop/src/features/agents/activeAgentTurnsStore.test.mjs‎

Lines changed: 214 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ import {
1111
saveActiveAgentTurnsForCommunity,
1212
restoreActiveAgentTurnsForCommunity,
1313
clearSavedCommunitySnapshot,
14+
clearActiveTurnsForAgent,
1415
} from "./activeAgentTurnsStore.ts";
1516
import {
1617
injectObserverEventsForE2E,
@@ -1737,3 +1738,216 @@ describe("community-switch save / restore", () => {
17371738
);
17381739
});
17391740
});
1741+
1742+
describe("clearActiveTurnsForAgent", () => {
1743+
const EPOCH = Date.parse("2024-01-01T00:00:00Z");
1744+
const at = (ms) => new Date(EPOCH + ms).toISOString();
1745+
1746+
beforeEach(() => {
1747+
resetActiveAgentTurnsStore();
1748+
});
1749+
1750+
it("clear removes the agent turns and notifies subscribers; other agents untouched", () => {
1751+
// Give AGENT two turns and AGENT_2 one turn.
1752+
syncAgentTurnsFromEvents(AGENT, [
1753+
makeEvent({ seq: 1, turnId: "t1", channelId: "c1" }),
1754+
makeEvent({ seq: 2, turnId: "t2", channelId: "c2" }),
1755+
]);
1756+
syncAgentTurnsFromEvents(AGENT_2, [
1757+
makeEvent({ seq: 1, turnId: "t3", channelId: "c3" }),
1758+
]);
1759+
1760+
let notified = 0;
1761+
const unsub = subscribeActiveAgentTurns(() => {
1762+
notified++;
1763+
});
1764+
clearActiveTurnsForAgent(AGENT);
1765+
unsub();
1766+
1767+
assert.equal(
1768+
getActiveTurnsForAgent(AGENT).length,
1769+
0,
1770+
"cleared agent must have no turns",
1771+
);
1772+
assert.equal(notified, 1, "must notify listeners exactly once");
1773+
1774+
// AGENT_2 is unaffected.
1775+
const a2channels = channelIdsOf(getActiveTurnsForAgent(AGENT_2));
1776+
assert.ok(a2channels.has("c3"), "other agent's turns must survive clear");
1777+
});
1778+
1779+
it("full-buffer replay after clear is a no-op (watermark preserved — badge stays gone)", () => {
1780+
// Process initial events to set the watermark at seq 2.
1781+
syncAgentTurnsFromEvents(AGENT, [
1782+
makeEvent({ seq: 1, turnId: "t1", channelId: "c1", timestamp: at(0) }),
1783+
makeEvent({
1784+
seq: 2,
1785+
turnId: "t2",
1786+
channelId: "c2",
1787+
timestamp: at(1_000),
1788+
}),
1789+
]);
1790+
clearActiveTurnsForAgent(AGENT);
1791+
assert.equal(
1792+
getActiveTurnsForAgent(AGENT).length,
1793+
0,
1794+
"should be empty after clear",
1795+
);
1796+
1797+
// Replay the identical buffer — every event is at or below the watermark.
1798+
let notified = 0;
1799+
const unsub = subscribeActiveAgentTurns(() => {
1800+
notified++;
1801+
});
1802+
syncAgentTurnsFromEvents(AGENT, [
1803+
makeEvent({ seq: 1, turnId: "t1", channelId: "c1", timestamp: at(0) }),
1804+
makeEvent({
1805+
seq: 2,
1806+
turnId: "t2",
1807+
channelId: "c2",
1808+
timestamp: at(1_000),
1809+
}),
1810+
]);
1811+
unsub();
1812+
1813+
assert.equal(
1814+
notified,
1815+
0,
1816+
"replay must not notify — watermark must be preserved",
1817+
);
1818+
assert.equal(
1819+
getActiveTurnsForAgent(AGENT).length,
1820+
0,
1821+
"badge must stay gone",
1822+
);
1823+
});
1824+
1825+
it("late turn_liveness frame with timestamp ≤ clear time does not resurrect (tombstone)", () => {
1826+
mock.timers.enable({ apis: ["Date"], now: EPOCH });
1827+
1828+
syncAgentTurnsFromEvents(AGENT, [
1829+
makeEvent({ seq: 1, turnId: "t1", channelId: "c1", timestamp: at(0) }),
1830+
]);
1831+
1832+
// Clear at EPOCH (t=0 in agent-host clock).
1833+
clearActiveTurnsForAgent(AGENT);
1834+
assert.equal(getActiveTurnsForAgent(AGENT).length, 0);
1835+
1836+
// A liveness frame whose timestamp is at or before the clear time must not
1837+
// resurrect the badge (tombstone blocks it). Advance seq past the
1838+
// watermark by using a higher seq than the initial turn_started.
1839+
syncAgentTurnsFromEvents(AGENT, [
1840+
makeEvent({
1841+
seq: 2,
1842+
kind: "turn_liveness",
1843+
turnId: "t1",
1844+
channelId: "c1",
1845+
timestamp: at(0), // equal to clear time — must NOT resurrect
1846+
}),
1847+
]);
1848+
1849+
assert.equal(
1850+
getActiveTurnsForAgent(AGENT).length,
1851+
0,
1852+
"liveness at or before clear time must not resurrect the cleared turn",
1853+
);
1854+
1855+
mock.timers.reset();
1856+
});
1857+
1858+
it("new turn_started after clear (restart picked up new work) is tracked normally", () => {
1859+
syncAgentTurnsFromEvents(AGENT, [
1860+
makeEvent({ seq: 1, turnId: "t1", channelId: "c1", timestamp: at(0) }),
1861+
]);
1862+
clearActiveTurnsForAgent(AGENT);
1863+
assert.equal(getActiveTurnsForAgent(AGENT).length, 0);
1864+
1865+
// A genuinely new turn arrives after the clear with a later timestamp and
1866+
// a new turnId — it must be tracked normally.
1867+
syncAgentTurnsFromEvents(AGENT, [
1868+
makeEvent({
1869+
seq: 2,
1870+
turnId: "t2",
1871+
channelId: "c1",
1872+
timestamp: at(5_000), // strictly newer than the cleared turn's timestamp
1873+
}),
1874+
]);
1875+
1876+
const turns = getActiveTurnsForAgent(AGENT);
1877+
assert.equal(turns.length, 1, "new turn after clear must be tracked");
1878+
assert.ok(channelIdsOf(turns).has("c1"), "new turn must surface c1");
1879+
});
1880+
1881+
it("badge is gone when stop succeeds even if start subsequently fails (stop-boundary clear)", () => {
1882+
// Arrange: agent has an active turn.
1883+
syncAgentTurnsFromEvents(AGENT, [
1884+
makeEvent({ seq: 1, turnId: "t1", channelId: "c1", timestamp: at(0) }),
1885+
]);
1886+
assert.equal(
1887+
getActiveTurnsForAgent(AGENT).length,
1888+
1,
1889+
"turn must be active before stop",
1890+
);
1891+
1892+
// Act: simulate what onStopped does — clear at the stop-success boundary,
1893+
// before start is called. Start fails (not called here).
1894+
clearActiveTurnsForAgent(AGENT);
1895+
1896+
// Assert: the badge is gone regardless of what happens to start.
1897+
assert.equal(
1898+
getActiveTurnsForAgent(AGENT).length,
1899+
0,
1900+
"badge must clear at stop-success boundary, not waiting for start to resolve",
1901+
);
1902+
});
1903+
1904+
it("new frame arriving while start is pending does not resurrect the cleared badge (tombstone boundary)", () => {
1905+
// Simulate: agent was active, stop succeeded and clear ran (onStopped
1906+
// fired), start is now in-flight. A stale liveness frame for the OLD
1907+
// turn arrives on the wire during the start-pending window. It must NOT
1908+
// resurrect the badge — the clear tombstoned it.
1909+
mock.timers.enable({ apis: ["Date"], now: EPOCH });
1910+
1911+
syncAgentTurnsFromEvents(AGENT, [
1912+
makeEvent({ seq: 1, turnId: "t1", channelId: "c1", timestamp: at(0) }),
1913+
]);
1914+
1915+
// onStopped fires: clear at stop boundary (agent-host clock = EPOCH).
1916+
clearActiveTurnsForAgent(AGENT);
1917+
1918+
// Stale liveness for t1 arrives with timestamp ≤ clear time (on-wire
1919+
// frame from before the kill). Must be blocked by the tombstone.
1920+
syncAgentTurnsFromEvents(AGENT, [
1921+
makeEvent({
1922+
seq: 2,
1923+
kind: "turn_liveness",
1924+
turnId: "t1",
1925+
channelId: "c1",
1926+
timestamp: at(0),
1927+
}),
1928+
]);
1929+
assert.equal(
1930+
getActiveTurnsForAgent(AGENT).length,
1931+
0,
1932+
"stale liveness during start-pending must not resurrect the cleared badge",
1933+
);
1934+
1935+
// Genuine new turn from the restarted agent arrives later with a new id
1936+
// and strictly newer timestamp — must be tracked normally.
1937+
syncAgentTurnsFromEvents(AGENT, [
1938+
makeEvent({
1939+
seq: 3,
1940+
turnId: "t2-new",
1941+
channelId: "c1",
1942+
timestamp: at(3_000),
1943+
}),
1944+
]);
1945+
assert.equal(
1946+
getActiveTurnsForAgent(AGENT).length,
1947+
1,
1948+
"genuine new turn from restarted agent must be tracked",
1949+
);
1950+
1951+
mock.timers.reset();
1952+
});
1953+
});

‎desktop/src/features/agents/activeAgentTurnsStore.ts‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -574,6 +574,36 @@ export function useActiveAgentTurnsBridge(
574574
}, [agents]);
575575
}
576576

577+
/**
578+
* Immediately clear all active turns for a specific agent — called when
579+
* Desktop itself stops or restarts the agent, so the turn store doesn't
580+
* have to wait for the 3-minute prune-pause backstop.
581+
*
582+
* Preserves `lastProcessed` (the watermark) so a full-buffer replay after
583+
* the clear is still a no-op — without the watermark a replayed
584+
* `turn_started` would immediately resurrect the badge. Preserves
585+
* `clockOffsetByAgent` — the offset remains valid and harmless.
586+
*
587+
* Tombstones every cleared turn (C) so an in-flight `turn_liveness` frame
588+
* already on the wire at kill time cannot resurrect the badge via
589+
* `resurrectTurn`. A restarted agent's genuinely new turns carry new
590+
* turnIds / newer timestamps, so the tombstones don't block them.
591+
*/
592+
export function clearActiveTurnsForAgent(agentPubkey: string): void {
593+
const key = normalizePubkey(agentPubkey);
594+
const agentTurns = activeTurnsByAgent.get(key);
595+
if (!agentTurns || agentTurns.size === 0) return;
596+
597+
const agentClockNow = Date.now() - (clockOffsetByAgent.get(key) ?? 0);
598+
for (const turnId of agentTurns.keys()) {
599+
recordTerminal(key, turnId, agentClockNow);
600+
}
601+
602+
activeTurnsByAgent.delete(key);
603+
invalidateCache(key);
604+
notifyListeners();
605+
}
606+
577607
/**
578608
* Clears all live turn state (active turns, offsets, watermarks, tombstones).
579609
* Intentionally preserves `savedByCommunity` — community-switch snapshots

‎desktop/src/features/agents/lib/managedAgentControlActions.test.mjs‎

Lines changed: 90 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,10 @@
11
import assert from "node:assert/strict";
22
import test from "node:test";
33

4-
import { startManagedAgentWithRules } from "./managedAgentControlActions.ts";
4+
import {
5+
startManagedAgentWithRules,
6+
respawnManagedAgentWithRules,
7+
} from "./managedAgentControlActions.ts";
58

69
function agent(overrides = {}) {
710
return {
@@ -77,3 +80,89 @@ test("ordinary local agents still start normally", async () => {
7780
});
7881
assert.equal(calledWith, "deadbeef".repeat(8));
7982
});
83+
84+
// --- respawnManagedAgentWithRules: stop→clear→start boundary tests -----------
85+
86+
test("test_respawn_stop_success_start_failure_onStopped_still_fires", async () => {
87+
// Prove: onStopped fires at the stop-success boundary even when start later
88+
// throws. This is the key discriminator: on round-1 code the clear only
89+
// ran after the full respawn, so a failed start left the badge intact.
90+
const runningAgent = agent({ status: "running" });
91+
let onStoppedFired = false;
92+
93+
await assert.rejects(
94+
respawnManagedAgentWithRules({
95+
agent: runningAgent,
96+
stopManagedAgent: async () => {
97+
/* stop succeeds */
98+
},
99+
startManagedAgent: async () => {
100+
throw new Error("start failed");
101+
},
102+
onStopped: () => {
103+
onStoppedFired = true;
104+
},
105+
}),
106+
/start failed/,
107+
);
108+
109+
assert.ok(
110+
onStoppedFired,
111+
"onStopped must fire at stop-success boundary even when start subsequently fails",
112+
);
113+
});
114+
115+
test("test_respawn_stop_failure_onStopped_not_called", async () => {
116+
// Prove: onStopped does NOT fire when stop itself throws. Clearing on a
117+
// failed stop would remove a badge that is still legitimately active.
118+
const runningAgent = agent({ status: "running" });
119+
let onStoppedFired = false;
120+
121+
await assert.rejects(
122+
respawnManagedAgentWithRules({
123+
agent: runningAgent,
124+
stopManagedAgent: async () => {
125+
throw new Error("stop failed");
126+
},
127+
startManagedAgent: async () => {
128+
/* should not be reached */
129+
},
130+
onStopped: () => {
131+
onStoppedFired = true;
132+
},
133+
}),
134+
/stop failed/,
135+
);
136+
137+
assert.ok(
138+
!onStoppedFired,
139+
"onStopped must NOT fire when stop itself fails — badge is still active",
140+
);
141+
});
142+
143+
test("test_respawn_onStopped_fires_before_start_resolves", async () => {
144+
// Prove: onStopped fires strictly between stop resolution and start
145+
// invocation. A clear that fires after start begins can tombstone genuine
146+
// new turns from the freshly spawned process.
147+
const runningAgent = agent({ status: "running" });
148+
const events = [];
149+
150+
await respawnManagedAgentWithRules({
151+
agent: runningAgent,
152+
stopManagedAgent: async () => {
153+
events.push("stop");
154+
},
155+
startManagedAgent: async () => {
156+
events.push("start");
157+
},
158+
onStopped: () => {
159+
events.push("onStopped");
160+
},
161+
});
162+
163+
assert.deepEqual(
164+
events,
165+
["stop", "onStopped", "start"],
166+
"onStopped must fire after stop resolves and before start is called",
167+
);
168+
});

‎desktop/src/features/agents/lib/managedAgentControlActions.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,13 +92,18 @@ export async function respawnManagedAgentWithRules({
9292
agent,
9393
startManagedAgent,
9494
stopManagedAgent,
95+
onStopped,
9596
}: {
9697
agent: ManagedAgent;
9798
startManagedAgent: StartManagedAgent;
9899
stopManagedAgent: StopManagedAgent;
100+
/** Called after a successful stop and before start begins — use this to
101+
* clear stale working badges at the right boundary. */
102+
onStopped?: () => void;
99103
}) {
100104
if (agent.backend.type === "local" && isManagedAgentActive(agent)) {
101105
await stopManagedAgent(agent.pubkey);
106+
onStopped?.();
102107
}
103108

104109
await startManagedAgent(agent.pubkey);

‎desktop/src/features/agents/lib/useAutoRestartPolicy.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ import {
55
managedAgentsQueryKey,
66
useManagedAgentsQuery,
77
} from "@/features/agents/hooks";
8+
import { clearActiveTurnsForAgentOnStop } from "@/features/agents/managedAgentRuntimeHooks";
89
import {
910
startManagedAgent,
1011
stopManagedAgent,
@@ -109,6 +110,7 @@ export function useAutoRestartPolicy() {
109110
return;
110111
}
111112
await stopManagedAgent(agent.pubkey);
113+
clearActiveTurnsForAgentOnStop(agent.pubkey);
112114
await startManagedAgent(agent.pubkey);
113115
} catch {
114116
// Failed attempt: edge stays consumed — badge-only until the

0 commit comments

Comments
 (0)