Skip to content

Commit 16e0757

Browse files
authored
feat(daemon): opt-in inactivity expiry for idle local device claims (#2833) (#2988)
* feat(daemon): opt-in inactivity expiry for idle local device claims #2833: on a host shared by several agents, a session that never runs `close` keeps its host-global device claim until the daemon stops, and every other agent reads DEVICE_IN_USE with no way to tell active from abandoned. Set AGENT_DEVICE_SESSION_IDLE_TIMEOUT_MS to expire a claim-holding session with no remote lease and no capture running for it, once it has taken no attached commands for that long. The daemon that owns the session settles it under that session's own execution lock pair, releases the claim, and leaves a bounded marker so the next command answers SESSION_NOT_FOUND with details.reason SESSION_IDLE_EXPIRED naming the window and the released device. Off by default; a positive window below one millisecond is the shortest window, not off. A settle that cannot confirm its claim gone holds the session record back and retries a window later: this daemon stays alive, so forgetting the record would strand a claim owned by a process that no longer knows what it holds, which `device release --stale` cannot reclaim. `clearDeviceClaim` therefore gains `unattributable` for a record that yields no attributable owner, because that says nothing about a successor having taken the device. The lock pair belongs to the release, not to the sweep's wait for it. A teardown budget bounds only that wait, so one stuck recorder cannot hang a sweep while a budget releasing the locks would let a retried `close` join a teardown in progress, or a late release delete a session a retried `open` just created and mark it expired. A sweep that finds a release in flight defers that address rather than re-arming at a deadline already past, and a release that lands after its budget still reports the session it freed. * chore(gates): declare lastActivityAtMs as a store-owned SessionState field R7 holds every SessionState field to one declared writer. The #2833 request path reports activity through `SessionStore.noteSessionActivity` rather than mutating the record itself, so the store that owns the record is the only writer here and the field now says so.
1 parent 70a6b55 commit 16e0757

33 files changed

Lines changed: 3347 additions & 62 deletions

‎scripts/layering/session-state.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -125,6 +125,9 @@ export const STORE_OWNED_SESSION_STATE_FIELDS: ReadonlySet<string> = new Set([
125125
'audioProbe',
126126
'createdAt',
127127
'device',
128+
// #2833: the request path reports session activity through `SessionStore.noteSessionActivity`, so
129+
// the only writer of this field is the store that owns the record.
130+
'lastActivityAtMs',
128131
'lastPerfProfile',
129132
'name',
130133
'recordOnlySession',

‎src/cli/commands/__tests__/daemon.test.ts‎

Lines changed: 46 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ const GRACEFUL_RESULT: DaemonStopResult = {
3333
claimsReleased: [],
3434
claimsOrphaned: [],
3535
claimsSuperseded: [],
36+
claimsUnattributable: [],
3637
providerReleases: { status: 'completed', released: [], pending: [] },
3738
warnings: [],
3839
};
@@ -77,7 +78,7 @@ test('merges a graceful shutdown report and cleans runner leases with the start-
7778
released: [{ leaseId: 'lease-1', provider: 'limrun' }],
7879
pending: [],
7980
},
80-
claims: { released: [claim], orphaned: [], superseded: [] },
81+
claims: { released: [claim], orphaned: [], superseded: [], unattributable: [] },
8182
});
8283

8384
try {
@@ -104,6 +105,7 @@ test('merges a graceful shutdown report and cleans runner leases with the start-
104105
claimsReleased: [claim],
105106
claimsOrphaned: [],
106107
claimsSuperseded: [],
108+
claimsUnattributable: [],
107109
}),
108110
expect.any(Function),
109111
);
@@ -152,7 +154,7 @@ test('warns in text output when a graceful stop leaves an orphaned claim', async
152154
};
153155
mocks.readDaemonShutdownReport.mockReturnValue({
154156
providerReleases: { released: [], pending: [] },
155-
claims: { released: [], orphaned: [claim], superseded: [] },
157+
claims: { released: [], orphaned: [claim], superseded: [], unattributable: [] },
156158
});
157159

158160
try {
@@ -176,3 +178,45 @@ test('warns in text output when a graceful stop leaves an orphaned claim', async
176178
fs.rmSync(stateDir, { recursive: true, force: true });
177179
}
178180
});
181+
182+
test('routes an unattributable claim to the default status view, never to --stale', async () => {
183+
const stateDir = mkdtempForTestSync('agent-device-daemon-command-');
184+
mocks.readDaemonStopIdentity.mockReturnValue({ pid: 123, processStartTime: 'start-time' });
185+
mocks.stopDaemon.mockResolvedValue(GRACEFUL_RESULT);
186+
const claim = {
187+
deviceKey: 'local:android:none:emulator-5554',
188+
session: 'default',
189+
platform: 'android',
190+
deviceId: 'emulator-5554',
191+
};
192+
mocks.readDaemonShutdownReport.mockReturnValue({
193+
providerReleases: { released: [], pending: [] },
194+
claims: { released: [], orphaned: [], superseded: [], unattributable: [claim] },
195+
});
196+
197+
try {
198+
await daemonCommand({
199+
positionals: ['stop'],
200+
flags: { clean: false, help: false, json: false, stateDir, version: false },
201+
client: {} as never,
202+
});
203+
204+
const [, data, renderHuman] = mocks.writeCommandOutput.mock.calls.at(-1) ?? [];
205+
expect(data).toEqual(
206+
expect.objectContaining({
207+
claimsUnattributable: [claim],
208+
claimsOrphaned: [],
209+
warnings: [expect.stringContaining('whose owner could not be read')],
210+
}),
211+
);
212+
const rendered = (renderHuman as () => string)();
213+
expect(rendered).toContain('Inspect with: agent-device device status.');
214+
// The whole reason this is its own bucket: both stale routes refuse a record with no recorded
215+
// owner, so the warning may name them only to forbid them and must not end in a release.
216+
expect(rendered).toContain('No device release route settles it');
217+
expect(rendered).not.toContain('then release with');
218+
expect(rendered).not.toContain('Inspect with: agent-device device status --stale');
219+
} finally {
220+
fs.rmSync(stateDir, { recursive: true, force: true });
221+
}
222+
});

‎src/cli/commands/daemon.ts‎

Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,10 +46,12 @@ function mergeShutdownReport(
4646
claimsReleased: report.claims.released,
4747
claimsOrphaned: report.claims.orphaned,
4848
claimsSuperseded: report.claims.superseded,
49+
claimsUnattributable: report.claims.unattributable,
4950
warnings: [
5051
...stopped.warnings,
5152
...supersededClaimWarnings(report.claims.superseded),
5253
...orphanedClaimWarnings(report.claims.orphaned),
54+
...unattributableClaimWarnings(report.claims.unattributable),
5355
],
5456
}
5557
: stopped;
@@ -77,7 +79,11 @@ function supersededClaimWarnings(superseded: DaemonStopResult['claimsSuperseded'
7779

7880
/** An orphaned claim keeps holding its device after the daemon is gone, and
7981
* only `device release --stale` or the next open settles it — say so instead
80-
* of leaving the block discoverable through --json alone. */
82+
* of leaving the block discoverable through --json alone.
83+
*
84+
* A record whose owner could not be attributed is NOT reported here. Those are what
85+
* `--stale` hides and refuses, so this sentence would send the operator to a route that cannot
86+
* succeed; {@link unattributableClaimWarnings} names the view that does show them. */
8187
function orphanedClaimWarnings(orphaned: DaemonStopResult['claimsOrphaned']): string[] {
8288
if (orphaned.length === 0) return [];
8389
const devices = orphaned.map((claim) => claim.deviceId).join(', ');
@@ -86,6 +92,27 @@ function orphanedClaimWarnings(orphaned: DaemonStopResult['claimsOrphaned']): st
8692
];
8793
}
8894

95+
/**
96+
* What the three provable buckets each get an exact remedy for, this one does not: settling a claim
97+
* always rests on a proof about its owner, and this record names none. So the warning states only
98+
* what is known, points at the view that lists these records — the default one, because `--stale`
99+
* filters records without a decodable owner out — and does not promise a release that would clear
100+
* them. `device release --stale` refuses every record that names no owner, and the next `open` refuses
101+
* to overwrite one. The one principal that can clear an allocator-held record is the allocator that
102+
* issued it, which is named on the refusal rather than guessed at here.
103+
*/
104+
function unattributableClaimWarnings(
105+
unattributable: DaemonStopResult['claimsUnattributable'],
106+
): string[] {
107+
if (unattributable.length === 0) return [];
108+
const devices = unattributable.map((claim) => claim.deviceId).join(', ');
109+
return [
110+
`A claim for ${devices} remains whose owner could not be read, so this daemon cannot say whether those devices are free. ` +
111+
'No device release route settles it: --stale proves staleness from the recorded owner and refuses records that name none, ' +
112+
'and open refuses to overwrite them. Inspect with: agent-device device status.',
113+
];
114+
}
115+
89116
function renderDaemonStop(
90117
result: Pick<DaemonStopResult, 'stopped' | 'mode' | 'warnings'> & {
91118
clean: boolean;

‎src/commands/schema/cli-help-topics.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -517,6 +517,7 @@ test('usageForCommand resolves physical-device help topic', async () => {
517517
);
518518
assert.match(help, /AGENT_DEVICE_IOS_RUNNER_IDLE_STOP_MS/);
519519
assert.match(help, /AGENT_DEVICE_DAEMON_IDLE_TIMEOUT_MS/);
520+
assert.match(help, /AGENT_DEVICE_SESSION_IDLE_TIMEOUT_MS/);
520521
assert.match(
521522
help,
522523
/a stale iOS runner lease — its owner process dead, or its AGENT_DEVICE_STATE_DIR deleted — is reclaimed automatically/i,

‎src/commands/schema/cli-help.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -702,6 +702,7 @@ Runner and daemon lifecycle (applies to simulators too):
702702
No runner read launches a session app that is not running: snapshot, wait, is, get, a reading find, and an interaction's leading reads (a gesture's viewport read, the capture that resolves a selector click/fill) answer the retriable APP_NOT_RUNNING instead of bare-launching over a launch SpringBoard still holds behind its deep-link confirmation. Only open, activate, and a command that mutates without a leading read bring a stopped app up.
703703
close keeps a healthy iOS simulator XCTest runner warm by default so the next open on that simulator (same udid in the same simulator set) skips the runner build, unless --shutdown was requested, the session was recording, or the session held a device lease. A retained runner auto-stops after an idle window (default 5 minutes); set AGENT_DEVICE_IOS_RUNNER_IDLE_STOP_MS to override, or 0 to disable idle stop and retain until daemon exit.
704704
Each AGENT_DEVICE_STATE_DIR runs its own daemon. It self-exits after an idle window (default 5 minutes, matching the runner idle-stop default) once it has no open sessions, no in-flight requests, and no active recording; set AGENT_DEVICE_DAEMON_IDLE_TIMEOUT_MS to override, or 0 to disable idle reap.
705+
On a machine shared by several agents, set AGENT_DEVICE_SESSION_IDLE_TIMEOUT_MS to also expire an individual session that has taken a host-global device claim and then received no commands for that long; the claim is released and the next command on that session answers SESSION_NOT_FOUND with details.reason SESSION_IDLE_EXPIRED naming the window and the device. Off by default, because on a shared host "idle" and "thinking" are indistinguishable: only set it where an abandoned claim blocking every other agent is the worse failure. Sessions holding a remote lease or any active capture (recording, logs, audio, performance, or trace) are never expired this way.
705706
A stale iOS runner lease — its owner process dead, or its AGENT_DEVICE_STATE_DIR deleted — is reclaimed automatically instead of failing with "is already owned by another agent-device daemon". A live owner's runner is also reclaimed when the requesting daemon holds the host-global device claim for that device: claims are exclusive, so holding one proves the runner's owner released the device and merely kept the runner warm. The error remains only for owners outside claim arbitration (a pre-claims build, or daemons pointed at different claim stores).
706707
707708
For iOS SpringBoard, widget, or other system-UI surfaces, read agent-device help ios-system-ui.`,

‎src/daemon/__tests__/application-lifecycle-recovery.test.ts‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -188,6 +188,30 @@ test('daemon lifecycle finalization admits facts once, binds once, and disposes
188188
expect(dispose).toHaveBeenCalledOnce();
189189
});
190190

191+
// #2833: an idle-session expiry is the one daemon-owned teardown with no successor to hand a healthy
192+
// execution host to. Deferring runner termination to the gateway's shutdown phase would park the
193+
// runner — and the device behind it — until process exit, which is the opposite of why the expiry ran.
194+
test('a daemon that stays alive finalizes without the shutdown deferral', async () => {
195+
const finalize = vi.fn(async () => {});
196+
const runtime = gateway({ finalize });
197+
198+
await finalizeDaemonSessionApplicationLifecycle({
199+
gateway: runtime.gateway,
200+
scope: scope(),
201+
session,
202+
stateDir: '/state',
203+
runtimeHints: {},
204+
daemonLeaving: false,
205+
});
206+
207+
expect(finalize).toHaveBeenCalledWith({
208+
appBundleId: undefined,
209+
surface: 'app',
210+
retainRunner: false,
211+
stateDir: '/state',
212+
});
213+
});
214+
191215
test.each([
192216
{
193217
name: 'Android provider device',

‎src/daemon/__tests__/daemon-shutdown-report.test.ts‎

Lines changed: 33 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -27,15 +27,15 @@ test('round-trips provider release and device claim records without lease creden
2727
try {
2828
writeDaemonShutdownReport(stateDir, {
2929
providerReleases: { released: [lease], pending: [lease] },
30-
claims: { released: [claim], orphaned: [], superseded: [claim] },
30+
claims: { released: [claim], orphaned: [], superseded: [claim], unattributable: [] },
3131
});
3232

3333
expect(readDaemonShutdownReport(stateDir)).toEqual({
3434
providerReleases: {
3535
released: [{ leaseId: lease.leaseId, provider: 'limrun' }],
3636
pending: [{ leaseId: lease.leaseId, provider: 'limrun' }],
3737
},
38-
claims: { released: [claim], orphaned: [], superseded: [claim] },
38+
claims: { released: [claim], orphaned: [], superseded: [claim], unattributable: [] },
3939
});
4040
} finally {
4141
fs.rmSync(stateDir, { recursive: true, force: true });
@@ -54,7 +54,7 @@ test('a report written before claim reporting still reads its provider releases'
5454

5555
expect(readDaemonShutdownReport(stateDir)).toEqual({
5656
providerReleases: { released: [], pending: [] },
57-
claims: { released: [], orphaned: [], superseded: [] },
57+
claims: { released: [], orphaned: [], superseded: [], unattributable: [] },
5858
});
5959
} finally {
6060
fs.rmSync(stateDir, { recursive: true, force: true });
@@ -82,3 +82,33 @@ test('ignores malformed shutdown reports and clear removes a prior report', () =
8282
fs.rmSync(stateDir, { recursive: true, force: true });
8383
}
8484
});
85+
86+
test('a report written before unattributable claims were separated still reads its three buckets', () => {
87+
const stateDir = mkdtempForTestSync('agent-device-shutdown-report-');
88+
const reportPath = path.join(stateDir, 'daemon-shutdown.json');
89+
const claim = {
90+
deviceKey: 'local:android:none:emulator-5554',
91+
session: 'default',
92+
platform: 'android',
93+
deviceId: 'emulator-5554',
94+
};
95+
96+
try {
97+
// The shape a previous daemon writes. `unattributable` is additive, so a reader of a report from
98+
// before it existed must still get every bucket it can describe rather than dropping the section.
99+
fs.writeFileSync(
100+
reportPath,
101+
JSON.stringify({
102+
providerReleases: { released: [], pending: [] },
103+
claims: { released: [claim], orphaned: [claim], superseded: [claim] },
104+
}),
105+
);
106+
107+
expect(readDaemonShutdownReport(stateDir)).toEqual({
108+
providerReleases: { released: [], pending: [] },
109+
claims: { released: [claim], orphaned: [claim], superseded: [claim], unattributable: [] },
110+
});
111+
} finally {
112+
fs.rmSync(stateDir, { recursive: true, force: true });
113+
}
114+
});

‎src/daemon/__tests__/filesystem-boundary-faults.test.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -138,7 +138,7 @@ function createShutdownReportFixture(root: string): FilesystemBoundaryFixture {
138138
run: async () =>
139139
writeDaemonShutdownReport(root, {
140140
providerReleases: { released: [], pending: [] },
141-
claims: { released: [], orphaned: [], superseded: [] },
141+
claims: { released: [], orphaned: [], superseded: [], unattributable: [] },
142142
}),
143143
expected: 'return',
144144
};

0 commit comments

Comments
 (0)