Skip to content

Commit a9018d1

Browse files
committed
revert(daemon): keep the liveness probe's HTTP leg standalone
Reverts the health-read reuse from c71b306. Both ways of breaking the static cycle the reuse creates fail a gate: - Fallow caught the cycle itself: `readDaemonHttpHealth` lives in the transport, the transport imports the timeout handler, and the timeout handler loads the probe on demand, so the probe importing the transport is transport -> timeout -> probe -> transport. - The two hosts that break it cost more than the ~25 lines they save. A new module adds one to the CLI's eager closure (295 -> 296), which the ADR 0019 loading-shape gate refuses; the only already-eager host is the wire-codec module, and hosting a /health reader there splits this PR across six of the ADR 0006 health declarations, digests and acks included. The probe's leg therefore builds its own /health GET again: it needs an absolute 1s budget and a cancel handle, where the reachability reader wants a 500ms cap and a reachability verdict. A new test pins the difference, because reading `reachable` in the probe would make a daemon answering 5xx look silent and authorize the kill #3177 is about. Keeps the parts of c71b306 that stand on their own: the comment trims from the same review, and the probe header's over-claim corrected — the idle-timeout argument holds for the socket leg, not for the HTTP reader's own cap.
1 parent a1ae1cf commit a9018d1

4 files changed

Lines changed: 41 additions & 47 deletions

File tree

‎src/daemon-client/__tests__/daemon-client-liveness-probe.test.ts‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -78,9 +78,10 @@ test('a daemon answering /health is responsive', async (t) => {
7878

7979
test('a 5xx answer is still an answer: the finding is liveness, not reachability policy', async (t) => {
8080
if (await skipWhenLoopbackUnavailable(t)) return;
81-
// The HTTP leg rides the reachability reader (`readDaemonHttpHealth`), which reports a 5xx as
82-
// UNREACHABLE for its own policy. Reading that flag here would make a daemon that answered 500
83-
// look silent and authorize killing a live shared daemon — the bug #3177 is about.
81+
// `readDaemonHttpHealth` (the reachability reader the transport uses before each command) reports
82+
// a 5xx as UNREACHABLE — correct for its own policy, wrong for this one. The probe asks only
83+
// whether the endpoint answered, so a daemon serving a 500 is alive and must not be killed:
84+
// reading a reachability flag here would reproduce the bug #3177 is about.
8485
const server = http.createServer((_req, res) => {
8586
res.statusCode = 503;
8687
res.end('overloaded');

‎src/daemon-client/daemon-client-liveness-probe.ts‎

Lines changed: 28 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,7 @@
11
import net from 'node:net';
22
import { INTERNAL_COMMANDS } from '@agent-device/command-registry/catalog';
33
import { createRequestId } from '@agent-device/host-kit/diagnostics';
4-
import { consumeTextLines } from '@agent-device/host-kit/transport';
5-
import { readDaemonHttpHealth } from './daemon-client-transport.ts';
4+
import { loadNodeHttpRequester, consumeTextLines } from '@agent-device/host-kit/transport';
65
import type { DaemonRequest } from '../daemon/daemon-request.ts';
76
import type { DaemonInfo } from './daemon-client-metadata.ts';
87

@@ -18,24 +17,24 @@ import type { DaemonInfo } from './daemon-client-metadata.ts';
1817
export const LIVENESS_PROBE_BUDGET_MS = 1_000;
1918

2019
/**
21-
* Whether this local daemon answers anything within the probe budget: its HTTP health route, or one
22-
* sessionless RPC on its socket. An answer on either leg proves the event loop serves fresh
23-
* requests; a refused or silent leg answers "not this one", not "not the daemon".
20+
* Whether this local daemon answers anything within the probe budget: its HTTP health route, or
21+
* one sessionless RPC on its socket. An answer on either leg proves the daemon's event loop serves
22+
* fresh requests; a refused or silent leg answers "not this one", not "not the daemon".
2423
*
2524
* Both legs run CONCURRENTLY against one shared deadline: the finding is per-daemon, not per-leg,
26-
* and a leg that hangs must not spend the window the other needs. The first AFFIRMATIVE is
27-
* unrevocable and settles at once, retiring the other leg. The deadline is ABSOLUTE, so a leg whose
28-
* endpoint trickles bytes still answers inside the window. A leg never rejects: a probe that cannot
29-
* ask is an endpoint that did not answer. `session` rides along so an isolation-scoped daemon routes
30-
* the probe like the request it follows.
25+
* and a leg that hangs must not spend the window the other leg needs. The first AFFIRMATIVE is
26+
* unrevocable and settles at once. The deadline is ABSOLUTE — a leg whose endpoint trickles bytes
27+
* would outlive an idle timeout, and the negative verdict must land inside the window. A leg never
28+
* rejects: a probe that cannot ask is an endpoint that did not answer. `session` rides along so an
29+
* isolation-scoped daemon routes the probe like the request it follows.
3130
*/
3231
export async function probeDaemonResponsive(
3332
info: DaemonInfo,
3433
params: Readonly<{ session?: string }> = {},
3534
): Promise<boolean> {
3635
const deadlineAtMs = Date.now() + LIVENESS_PROBE_BUDGET_MS;
3736
const legs: ProbeLeg[] = [];
38-
if (info.httpPort) legs.push(probeHttpHealth(info, deadlineAtMs));
37+
if (info.httpPort) legs.push(probeHttpHealth(info.httpPort, deadlineAtMs));
3938
if (info.port) legs.push(probeSocketRpc(info.port, info.token, params.session, deadlineAtMs));
4039
if (legs.length === 0) return false;
4140
return await new Promise<boolean>((resolve) => {
@@ -85,27 +84,31 @@ function createProbeLeg(deadlineAtMs: number): {
8584
};
8685
}
8786

88-
function probeHttpHealth(info: DaemonInfo, deadlineAtMs: number): ProbeLeg {
87+
function probeHttpHealth(httpPort: number, deadlineAtMs: number): ProbeLeg {
8988
const probe = createProbeLeg(deadlineAtMs);
90-
const abort = new AbortController();
91-
// `readDaemonHttpHealth` owns the /health request; the probe supplies its own absolute budget and
92-
// a way to retire the request once the other leg decides the finding. A malformed record throws
93-
// inside that function — an endpoint that did not answer, never a rejection from the recovery path.
89+
// `readDaemonInfo` accepts any positive integer port, and `transport.request` throws on one out
90+
// of range; this leg builds detached, so the throw would reach the caller as an unhandled
91+
// rejection.
9492
void (async () => {
9593
try {
96-
const health = await readDaemonHttpHealth(info, Math.max(1, deadlineAtMs - Date.now()), {
97-
signal: abort.signal,
98-
budgetOverridesHealthCheckCap: true,
99-
});
100-
// ANY status is an answer (the health route is served ahead of request handling, so reaching
101-
// it proves the event loop serves fresh requests). Reading `reachable` instead would let a
102-
// 5xx read as silence and authorize killing a daemon that was answering.
103-
probe.settle(health.statusCode !== undefined);
94+
const transport = await loadNodeHttpRequester('http:');
95+
const request = transport.request(
96+
{ host: '127.0.0.1', port: String(httpPort), path: '/health', method: 'GET' },
97+
(res) => {
98+
// Any status is an answer: the health route is served before any request handling, so
99+
// reaching it proves the event loop serves fresh requests. Drain so the response cannot
100+
// hold the socket open past the finding.
101+
res.resume();
102+
probe.settle(true);
103+
},
104+
);
105+
request.on('error', () => probe.settle(false));
106+
probe.attach(() => request.destroy());
107+
request.end();
104108
} catch {
105109
probe.settle(false);
106110
}
107111
})();
108-
probe.attach(() => abort.abort());
109112
return probe.leg;
110113
}
111114

‎src/daemon-client/daemon-client-transport.ts‎

Lines changed: 6 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -141,16 +141,9 @@ export async function readRemoteDaemonHealth(
141141
return health;
142142
}
143143

144-
export async function readDaemonHttpHealth(
144+
async function readDaemonHttpHealth(
145145
info: DaemonInfo,
146146
probeTimeoutMs?: number,
147-
/**
148-
* `signal` lets a caller that races this health read against another probe retire the request
149-
* once the race is decided. `budgetOverridesHealthCheckCap` opts out of the health-check timeouts,
150-
* which are a policy for reachability probes: the post-timeout liveness probe (#3177) carries its
151-
* own absolute budget, and truncating it to 500ms would make a slow-to-answer daemon read as dead.
152-
*/
153-
options: Readonly<{ signal?: AbortSignal; budgetOverridesHealthCheckCap?: boolean }> = {},
154147
): Promise<RemoteDaemonHealth> {
155148
const endpoint = info.baseUrl
156149
? buildDaemonHttpUrl(info.baseUrl, 'health')
@@ -160,15 +153,12 @@ export async function readDaemonHttpHealth(
160153
if (!endpoint) return { reachable: false };
161154
const url = new URL(endpoint);
162155
const transport = await loadNodeHttpRequester(url.protocol);
163-
const healthCheckCapMs = info.baseUrl
164-
? REMOTE_DAEMON_HEALTHCHECK_TIMEOUT_MS
165-
: LOCAL_DAEMON_HEALTHCHECK_TIMEOUT_MS;
166-
const timeoutMs =
167-
options.budgetOverridesHealthCheckCap && probeTimeoutMs !== undefined
168-
? probeTimeoutMs
169-
: Math.min(healthCheckCapMs, probeTimeoutMs ?? Number.POSITIVE_INFINITY);
156+
const timeoutMs = Math.min(
157+
info.baseUrl ? REMOTE_DAEMON_HEALTHCHECK_TIMEOUT_MS : LOCAL_DAEMON_HEALTHCHECK_TIMEOUT_MS,
158+
probeTimeoutMs ?? Number.POSITIVE_INFINITY,
159+
);
170160
if (timeoutMs <= 0) return { reachable: false, timedOut: true };
171-
const signal = options.signal ?? AbortSignal.timeout(Math.ceil(timeoutMs));
161+
const signal = AbortSignal.timeout(Math.ceil(timeoutMs));
172162
return await new Promise((resolve) => {
173163
const headers = info.baseUrl ? buildDaemonHttpAuthHeaders(info.token) : {};
174164
const unreachable = (): RemoteDaemonHealth =>

‎test/wire-compat/ledger.json‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@
6767
"src/daemon-client/daemon-client-rpc.ts#toDaemonHttpRpcError": "sha256:888246763c48670e7da893054d025744654f8715c3b4906312617a2b5028316b",
6868
"src/daemon-client/daemon-client-transport.ts#RemoteDaemonHealth": "sha256:3bac36fa97090b273afe1128103e1bf476d05441fd9de41317f1b78775b88304",
6969
"src/daemon-client/daemon-client-transport.ts#RemoteDaemonHealthLink": "sha256:7702598468b4c82b4ffa93064cd6bdfec938c1c527f7e6007c0584f3884a6eea",
70-
"src/daemon-client/daemon-client-transport.ts#readDaemonHttpHealth": "sha256:334280feb8adb487ddbd53c9b79bd837fd8fa7155fd124c5528192455d49fac1",
70+
"src/daemon-client/daemon-client-transport.ts#readDaemonHttpHealth": "sha256:eef0e153eb6f0bc02175e464dd6d98559b500b65ea8c74ba2203cfef09ec09a0",
7171
"src/daemon-client/daemon-client-transport.ts#readHealthLink": "sha256:f56404b94d73da57de7248719c9136b760d2be8b4a53cde5315a2311b088425b",
7272
"src/daemon-client/daemon-client-transport.ts#readHealthPayload": "sha256:4e85ffc3e35e02379c393e9312344757e003cf1f0ad9eb8d1f77d90c81c861f1",
7373
"src/daemon-client/daemon-client-transport.ts#readRemoteDaemonHealth": "sha256:bcefa89fbb7fcbd6fee1b5ecb217955b9eee8d6fd2ad175fb653011edbd199d9",
@@ -431,8 +431,8 @@
431431
},
432432
{
433433
"declaration": "src/daemon-client/daemon-client-transport.ts#readDaemonHttpHealth",
434-
"digest": "sha256:334280feb8adb487ddbd53c9b79bd837fd8fa7155fd124c5528192455d49fac1",
435-
"rationale": "The post-timeout liveness probe (#3177) reuses this health read as its HTTP leg, so the budget and abort signal became caller-supplied options instead of an internal AbortSignal.timeout. Both are client-local: the /health request line, method, headers and accepted payload fields a released daemon or proxy sees are unchanged, and the health-check timeout caps still apply to every reachability probe that does not opt out."
434+
"digest": "sha256:eef0e153eb6f0bc02175e464dd6d98559b500b65ea8c74ba2203cfef09ec09a0",
435+
"rationale": "A restart retry must tell a probe that ran out of time from one that failed, so the client can report the RPC deadline instead of 'Remote daemon is unavailable'. `timedOut` is client-local: the prober sets it on its own timeout and abort paths and never reads it from a /health payload. The health request and the accepted payload fields are unchanged, so a released daemon or proxy is probed and parsed exactly as before."
436436
}
437437
]
438438
}

0 commit comments

Comments
 (0)