Skip to content

Commit 0da0a66

Browse files
thymikeeclaude
andcommitted
fix(daemon-client): name the state dir in the not-running hint and report a force-retired drain honestly
Part of #3747, refs #3754 Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
1 parent ef16dde commit 0da0a66

8 files changed

Lines changed: 67 additions & 13 deletions

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
## Unreleased
44

5+
- The `daemon-not-running` hint now names `--state-dir <dir>` when the state directory is not the default, and a daemon force-retired for outlasting its drain is reported as `Replaced unreachable daemon` instead of `Replaced idle daemon`. `daemon-drain-timeout` and the post-drain force retire are documented in the client API and `help`.
56
- Fix `ensureDaemon()`, the CLI and MCP killing a daemon they had just asked to leave. After the daemon answers the quiesce request with `draining`, the client now waits up to 60 s for it to exit by itself and clean up its registration, so plugin disposal and lease finalization finish (on Windows a signal ends the process outright, and on POSIX a drain longer than 3 s was killed). A daemon still running at the deadline that fails its liveness probes is force-retired; one that still answers is left running and the call fails with `details.reason: 'daemon-drain-timeout'`, with a hint naming `agent-device daemon stop --state-dir <path>`.
67
- `@agent-device/testmu` is now a version 2 provider plugin. `import testmu from '@agent-device/testmu'` gives a factory, `testmu({ credentials: { username, accessKey } })`, that reads those credentials instead of `LT_USERNAME` and `LT_ACCESS_KEY`; without options it reads the variables as before. The package still declares the version 1 plugin entry, so `agent-device plugins add @agent-device/testmu` keeps working on hosts that predate the version 2 contract.
78
- Provider plugins have a new contract. A package exports a factory at `./agent-device` that returns `{ apiVersion: 2, provider, credentials, connection?, setup(host) }`, typed by `AgentDevicePluginFactory` from `agent-device/plugins`. The store calls the factory with the package's `config.json` options, and a JavaScript host can call it directly. Version 1 plugins (`agentDevicePlugin.apiVersion: 1`) keep loading.

‎packages/daemon/src/core/session-store.ts‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ import {
33
resolveSessionAppLogPath,
44
resolveSessionAppLogPidPath,
55
} from '@agent-device/daemon-contracts/session-artifact-paths';
6+
import { DAEMON_SHUTTING_DOWN_REASON } from '@agent-device/daemon-contracts/daemon-resolution';
67
import path from 'node:path';
78
import fs from 'node:fs';
89
import { AppError } from '@agent-device/kernel/errors';
@@ -74,7 +75,7 @@ export class SessionStore {
7475
assertAdmissionOpen(address: string): void {
7576
if (!this.acceptingSessions) {
7677
throw new AppError('COMMAND_FAILED', 'Daemon is shutting down', {
77-
reason: 'daemon_shutting_down',
78+
reason: DAEMON_SHUTTING_DOWN_REASON,
7879
session: address,
7980
});
8081
}

‎src/daemon-client/__tests__/daemon-client-ensure-drain.test.ts‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -179,9 +179,16 @@ test('a daemon still running past the deadline that answers no probe is force-re
179179
launchesFreshDaemonAt(spawn, paths, fresh.port);
180180

181181
try {
182-
const connection = await ensureDaemon({ stateDir, daemonTransport: 'http' });
182+
const started: Array<[string, number | undefined]> = [];
183+
const connection = await ensureDaemon({
184+
stateDir,
185+
daemonTransport: 'http',
186+
onStart: (reason, previous) => started.push([reason, previous?.pid]),
187+
});
183188

184189
assert.equal(connection.started, true);
190+
// Force-retired for outlasting its drain, it was never shown idle: reported as a missing daemon.
191+
assert.deepEqual(started, [['missing', childPid]]);
185192
assert.equal(drain.signals.length > 0, true, 'the stuck daemon was signaled');
186193
assert.ok(
187194
drain.signals.every(({ atMs }) => atMs >= DRAIN_DEADLINE_MS),

‎src/daemon-client/__tests__/daemon-client-explicit-lifecycle.test.ts‎

Lines changed: 37 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,10 @@ import path from 'node:path';
44
import { afterEach, test, vi } from 'vitest';
55
import { closeLoopbackServer, supportsLoopbackBind } from '@agent-device/test-kit';
66
import { AppError } from '@agent-device/kernel/errors';
7-
import { DAEMON_QUIESCE_COMMAND } from '@agent-device/daemon-contracts/daemon-resolution';
7+
import {
8+
DAEMON_QUIESCE_COMMAND,
9+
resolveDaemonPaths,
10+
} from '@agent-device/daemon-contracts/daemon-resolution';
811
import { DAEMON_STARTUP_EXIT_CODES } from '@agent-device/daemon-contracts/daemon-registration-owner';
912
import { isProcessAlive } from '@agent-device/host-kit/process';
1013
import { readVersion } from '@agent-device/host-kit/version';
@@ -39,6 +42,7 @@ import { sleep } from '@agent-device/host-kit/retry';
3942
import * as transport from '../daemon-client-transport.ts';
4043
import { sendToDaemon } from '../daemon-client.ts';
4144
import { ensureDaemon } from '../daemon-client-ensure.ts';
45+
import { daemonNotRunningError } from '../daemon-lifecycle-errors.ts';
4246

4347
// The explicit lifecycle: what an attach client may never do, and what `ensureDaemon` may do only
4448
// through the daemon's own answer. Daemons here are loopback stand-ins registered by real children,
@@ -107,10 +111,17 @@ test('attach reports a missing daemon without starting one', async (t) => {
107111
const error = await failureOf(sendToDaemon(request(stateDir), { daemonLifecycle: 'attach' }));
108112

109113
assert.equal(error.details?.reason, 'daemon-not-running');
114+
assert.match(String(error.details?.hint), /agent-device devices --state-dir /);
110115
assert.equal(spawn.mock.calls.length, 0);
111116
assert.equal(fs.existsSync(paths.infoPath), false);
112117
});
113118

119+
test('the not-running hint leaves out --state-dir for the default state directory', () => {
120+
const error = daemonNotRunningError(resolveDaemonPaths(undefined).baseDir);
121+
122+
assert.match(String(error.details?.hint), /\(agent-device devices, for example\)/);
123+
});
124+
114125
test('attach reports an incompatible daemon with both identities and leaves it running', async (t) => {
115126
if (!(await withLoopback(t))) return;
116127
const { stateDir, paths } = stateDirAndPaths();
@@ -316,6 +327,31 @@ test.for([
316327
}
317328
});
318329

330+
test('ensureDaemon tells a daemon that is still starting up apart from a busy one', async (t) => {
331+
if (!(await withLoopback(t))) return;
332+
const { stateDir, paths } = stateDirAndPaths();
333+
const stale = await startScriptedDaemon(
334+
busy({ startingUp: true, inFlightRequests: 0, sessions: 0, repairSessions: 0, leases: 0 }),
335+
);
336+
const { child } = await registerDaemon(paths, {
337+
httpPort: stale.port,
338+
version: '0.0.0-stale',
339+
capabilities: ['quiesce'],
340+
});
341+
342+
try {
343+
const error = await failureOf(ensureDaemon({ stateDir, daemonTransport: 'http' }));
344+
345+
assert.equal(error.details?.reason, 'daemon-busy');
346+
assert.equal(error.details?.startingUp, true);
347+
assert.match(String(error.details?.hint), /still starting up\. Retry in a moment/);
348+
assert.equal(isProcessAlive(child.pid), true);
349+
assert.equal(spawn.mock.calls.length, 0);
350+
} finally {
351+
await closeLoopbackServer(stale.server);
352+
}
353+
});
354+
319355
test('ensureDaemon takes a refusal from its typed reason, not from its wording', async (t) => {
320356
if (!(await withLoopback(t))) return;
321357
const { stateDir, paths } = stateDirAndPaths();

‎src/daemon-client/__tests__/explicit-lifecycle-fixture.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -106,7 +106,7 @@ export async function startScriptedSocketDaemon(
106106

107107
export const okResult = (data: Record<string, unknown>) => ({ ok: true, data });
108108
export const draining = okResult({ outcome: 'draining' });
109-
export const busy = (census: Record<string, number>) => ({
109+
export const busy = (census: Record<string, number | boolean>) => ({
110110
ok: false,
111111
error: {
112112
code: 'COMMAND_FAILED',

‎src/daemon-client/daemon-client-ensure.ts‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -127,10 +127,11 @@ async function replaceIncompatible(
127127
}
128128
const outcome = await quiesceOrRetireUnresponsive(info, settings);
129129
if (outcome === 'gone') return { kind: 'judge-again' };
130-
if (outcome === 'draining') await awaitDrainedDaemon(info, startTime, settings);
130+
const drained =
131+
outcome === 'draining' ? await awaitDrainedDaemon(info, startTime, settings) : outcome;
131132
// A daemon force-retired for answering nothing is reported like an unreachable one: it was never
132133
// shown to be idle, so "incompatible" would overstate what happened to it.
133-
const reason = outcome === 'retired' ? 'missing' : 'incompatible';
134+
const reason = drained === 'retired' ? 'missing' : 'incompatible';
134135
return await start(settings, onStart, reason, daemonIdentityOf(info));
135136
}
136137

@@ -149,7 +150,7 @@ async function awaitDrainedDaemon(
149150
info: DaemonInfo,
150151
startTime: string,
151152
settings: DaemonClientSettings,
152-
): Promise<void> {
153+
): Promise<'left' | 'retired'> {
153154
const { exited } = await waitForDaemonExit(
154155
{ pid: info.pid, startTime },
155156
{ timeoutMs: DRAIN_DEADLINE_MS, pollMs: DRAIN_POLL_MS },
@@ -160,11 +161,11 @@ async function awaitDrainedDaemon(
160161
paths: settings.paths,
161162
observed: { pid: info.pid, startTime },
162163
});
163-
return;
164+
return 'left';
164165
}
165166
if (await answersLivenessProbe(info)) throw await daemonDrainTimeoutError(info, settings);
166167
const retirement = await stopDaemonForTakeover(info, settings.paths, 'force');
167-
if (retirement.status !== 'retained') return;
168+
if (retirement.status !== 'retained') return 'retired';
168169
// The identity check could not prove the live process is the daemon it observed (an embedded host
169170
// runs no daemon entry; a start time or command line may also be unreadable), so nothing was
170171
// signaled.

‎src/daemon-client/daemon-client-ensuring.ts‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
import { AppError } from '@agent-device/kernel/errors';
21
import type { DaemonIdentity, EnsureDaemonOptions } from '@agent-device/contracts/client';
32
import { resolveDaemonPaths } from '@agent-device/daemon-contracts/daemon-resolution';
43
import type { sendToDaemon } from './daemon-client.ts';
4+
import { isDaemonNotRunningError } from './daemon-lifecycle-errors.ts';
55

66
type DaemonTransport = typeof sendToDaemon;
77

@@ -66,8 +66,7 @@ export function withEnsuredDaemon(
6666
} catch (error) {
6767
// The daemon this invocation ensured is gone (idle reap, a crash, another client's stop).
6868
// An attach that finds none sent nothing, so ensuring again and sending once is safe.
69-
if (!(error instanceof AppError) || error.details?.reason !== 'daemon-not-running')
70-
throw error;
69+
if (!isDaemonNotRunningError(error)) throw error;
7170
// Only the promise this attach relied on is dropped: a concurrent request may already have
7271
// replaced it with a fresh one.
7372
if (ensured.get(stateDir) === relied) ensured.delete(stateDir);

‎src/daemon-client/daemon-lifecycle-errors.ts‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import type { DaemonIdentity } from '@agent-device/contracts/client';
22
import {
33
DAEMON_QUIESCE_REASONS,
4+
resolveDaemonPaths,
45
type DaemonQuiesceCensus,
56
} from '@agent-device/daemon-contracts/daemon-resolution';
67
import { AppError } from '@agent-device/kernel/errors';
@@ -57,14 +58,22 @@ export function daemonContendedError(): AppError {
5758
}
5859

5960
export function daemonNotRunningError(stateDir: string, registered?: DaemonInfo): AppError {
61+
const stateDirFlag =
62+
stateDir === resolveDaemonPaths(undefined).baseDir
63+
? ''
64+
: ` --state-dir ${shellQuoteIfNeeded(stateDir)}`;
6065
return new AppError('COMMAND_FAILED', 'No daemon is running for this state directory.', {
6166
reason: DAEMON_LIFECYCLE_REASONS.notRunning,
6267
stateDir,
6368
...(registered ? { daemon: daemonIdentityOf(registered) } : {}),
64-
hint: `Start one with a local agent-device command that uses the daemon (devices, for example), or with ensureDaemon({ stateDir: ${JSON.stringify(stateDir)} }) from the SDK, before attaching.${registered ? ` The registered daemon (pid ${registered.pid}) does not answer; ${daemonStopCommand(stateDir)} clears it.` : ''}`,
69+
hint: `Start one with a local agent-device command that uses the daemon (agent-device devices${stateDirFlag}, for example), or with ensureDaemon({ stateDir: ${JSON.stringify(stateDir)} }) from the SDK, before attaching.${registered ? ` The registered daemon (pid ${registered.pid}) does not answer; ${daemonStopCommand(stateDir)} clears it.` : ''}`,
6570
});
6671
}
6772

73+
export function isDaemonNotRunningError(error: unknown): boolean {
74+
return error instanceof AppError && error.details?.reason === DAEMON_LIFECYCLE_REASONS.notRunning;
75+
}
76+
6877
export async function daemonIncompatibleError(params: {
6978
info: DaemonInfo;
7079
mismatch: DaemonTakeoverCause | 'newer' | 'no-quiesce';

0 commit comments

Comments
 (0)