Skip to content

Commit 9b90f76

Browse files
committed
test(cli): classify a top-level dynamic import as eager, and share one HTTP loader
"On demand" is a claim about SCOPE, not syntax, and the guard only knew syntax. `import('node:http')` at MODULE TOP LEVEL runs during module evaluation -- so a top-level `await import(...)`, an `import(...).then(...)`, or an immediately-invoked top-level function reintroduced the whole undici plus trust-store cost while the guard stayed green. A dynamic import is lazy only when its nearest enclosing function scope is not the module itself. The guard now walks each closure file's OXC AST carrying an `eager` flag that drops on entry to a FunctionDeclaration/FunctionExpression/ArrowFunction body, and reports `import()` calls still reached with it set. Immediately-invoked top-level functions are detected by unwrapping the callee's ParenthesizedExpression, so `(async () => { ... })()` is eager too. The one shape that still reads as lazy -- a top-level function invoked indirectly at load time -- is stated as a known limitation in the test rather than left implied. Verified red by inserting each shape into a REACHABLE closure module (daemon-client-transport.ts): top-level await, `.then(...)`, and the top-level IIFE each fail the guard and name the file. The lazy case is proven by the suite itself rather than a fixture alone -- the shared loader below lives in the closure, so its function-local `await import('node:http')` would turn the guard red if scope tracking were wrong. The three lazy-load sites had drifted into three copies of the same ternary. They now share `loadNodeHttpRequester` in src/utils/node-http.ts, next to the other node:http helpers, which is where the reason for the indirection is documented once. This also clears the Fallow complexity finding the previous push introduced. Fixture table extended to 19 cases covering both axes: static form (side-effect, default, namespace, named, value/star re-export, mixed, type-only x3) and dynamic scope (top-level x3, function/arrow/method-local, and the exact ternary the shipped loader uses).
1 parent 402dcb0 commit 9b90f76

5 files changed

Lines changed: 183 additions & 75 deletions

File tree

‎src/__tests__/cli-startup-import-closure.test.ts‎

Lines changed: 146 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -12,23 +12,36 @@ import { parseSync } from 'oxc-parser';
1212
* socket). The HTTP daemon transport, remote-artifact upload and download all
1313
* load these modules on demand instead.
1414
*
15-
* Classification comes from oxc-parser's ES module record rather than a regex,
16-
* because the forms that matter are exactly the ones a regex tuned to
17-
* `import ... from` misses: a bare side-effect `import 'node:http'` binds
18-
* nothing yet still evaluates the module, so it reintroduces the whole cost
19-
* while looking like nothing at all. Type-only imports are erased at build and
20-
* stay allowed; dynamic `import()` is the fix and is deliberately not followed.
15+
* "On demand" is a claim about SCOPE, not about syntax, which is why this reads
16+
* the AST rather than matching import forms. Two shapes evaluate the module
17+
* during module evaluation while looking lazy or looking like nothing at all:
18+
* a bare side-effect `import 'node:http'` (binds no names, still evaluates),
19+
* and a dynamic `import('node:http')` sitting at module top level rather than
20+
* inside a function -- including `.then(...)` and an immediately-invoked
21+
* top-level function. A dynamic import is lazy only when its nearest enclosing
22+
* function scope is not the module itself. Type-only imports and re-exports are
23+
* erased at build and stay allowed.
24+
*
25+
* Known limitation: a top-level function that is invoked indirectly at load
26+
* time (stored, then called by another top-level statement) reads as lazy here.
27+
* Direct top-level invocation -- `(() => { ... })()` -- is detected.
2128
*/
2229

2330
const srcRoot = path.resolve(import.meta.dirname, '..');
2431
const LAZY_HTTP_MODULES = new Set(['node:http', 'node:https']);
32+
const FUNCTION_NODES = new Set([
33+
'FunctionDeclaration',
34+
'FunctionExpression',
35+
'ArrowFunctionExpression',
36+
]);
37+
const WORKSPACE_SPECIFIER = /^(@agent-device\/[^/]+)(\/.*)?$/;
2538

26-
/**
27-
* Specifiers whose module this file causes to be EVALUATED at load time.
28-
* Excludes type-only imports/re-exports (erased) and dynamic imports (lazy).
29-
*/
30-
function evaluatedModuleRefs(fileName: string, source: string): string[] {
31-
const { module } = parseSync(fileName, source);
39+
type AstNode = { type?: string; [key: string]: unknown };
40+
41+
type ParsedModuleRecord = ReturnType<typeof parseSync>['module'];
42+
43+
/** Static specifiers this file causes to be EVALUATED (not type-only). */
44+
function staticEvaluatedRefs(module: ParsedModuleRecord): string[] {
3245
const refs: string[] = [];
3346
for (const staticImport of module.staticImports) {
3447
// No entries at all is a side-effect import (`import 'x'`), which always
@@ -45,6 +58,63 @@ function evaluatedModuleRefs(fileName: string, source: string): string[] {
4558
return refs;
4659
}
4760

61+
function unwrapParentheses(node: unknown): AstNode | null {
62+
let current = node as AstNode | null;
63+
while (current?.type === 'ParenthesizedExpression')
64+
current = current.expression as AstNode | null;
65+
return current;
66+
}
67+
68+
/** The body of a function invoked right where it is defined, if this is that call. */
69+
function immediatelyInvokedBody(node: AstNode): unknown {
70+
if (node.type !== 'CallExpression') return null;
71+
const callee = unwrapParentheses(node.callee);
72+
return callee && FUNCTION_NODES.has(String(callee.type)) ? callee.body : null;
73+
}
74+
75+
function dynamicImportSpecifier(node: AstNode): string | null {
76+
if (node.type !== 'ImportExpression') return null;
77+
const source = node.source as { type?: string; value?: unknown } | undefined;
78+
return source?.type === 'Literal' && typeof source.value === 'string' ? source.value : null;
79+
}
80+
81+
function recordDynamicImport(record: AstNode, eager: boolean, found: string[]): void {
82+
if (!eager) return;
83+
const specifier = dynamicImportSpecifier(record);
84+
if (specifier !== null) found.push(specifier);
85+
}
86+
87+
/** Descends into a node's children, dropping `eager` on the way into a function body. */
88+
function visitChildren(record: AstNode, eager: boolean, found: string[]): void {
89+
const childEager = eager && !FUNCTION_NODES.has(String(record.type));
90+
for (const [key, value] of Object.entries(record)) {
91+
if (key !== 'type') collectEagerDynamicImports(value, childEager, found);
92+
}
93+
}
94+
95+
/** Walks the AST, collecting only `import()` calls reached without entering a function. */
96+
function collectEagerDynamicImports(node: unknown, eager: boolean, found: string[]): void {
97+
if (Array.isArray(node)) {
98+
for (const child of node) collectEagerDynamicImports(child, eager, found);
99+
return;
100+
}
101+
if (!node || typeof node !== 'object') return;
102+
const record = node as AstNode;
103+
recordDynamicImport(record, eager, found);
104+
// An immediately-invoked function runs now, so its body inherits `eager`.
105+
const invokedBody = immediatelyInvokedBody(record);
106+
if (invokedBody) collectEagerDynamicImports(invokedBody, eager, found);
107+
visitChildren(record, eager, found);
108+
}
109+
110+
/** Every specifier this file evaluates at load time, static or dynamic. */
111+
function eagerlyEvaluatedModules(fileName: string, source: string): string[] {
112+
const parsed = parseSync(fileName, source);
113+
const dynamic: string[] = [];
114+
collectEagerDynamicImports(parsed.program, true, dynamic);
115+
return [...new Set([...staticEvaluatedRefs(parsed.module), ...dynamic])];
116+
}
117+
48118
function resolveRelative(fromFile: string, specifier: string): string | null {
49119
const candidate = path.resolve(path.dirname(fromFile), specifier);
50120
if (fs.existsSync(candidate) && fs.statSync(candidate).isFile()) return candidate;
@@ -67,22 +137,30 @@ function readWorkspacePackageDirs(): Map<string, string> {
67137
return dirs;
68138
}
69139

140+
type ExportTarget = { default?: string; types?: string } | string;
141+
142+
function readExportTarget(packageDir: string, subpath: string): string | undefined {
143+
const manifest = JSON.parse(fs.readFileSync(path.join(packageDir, 'package.json'), 'utf8')) as {
144+
exports?: Record<string, ExportTarget>;
145+
};
146+
const target = manifest.exports?.[subpath];
147+
if (typeof target === 'string') return target;
148+
return target?.default ?? target?.types;
149+
}
150+
70151
/**
71152
* Workspace subpath imports are followed too: a package the CLI evaluates can
72153
* pull `node:http` in just as effectively as a file under src/, and stopping the
73154
* walk at the package boundary would be the same blind spot in a new place.
74155
*/
75156
function resolveWorkspace(specifier: string, packageDirs: Map<string, string>): string | null {
76-
const match = /^(@agent-device\/[^/]+)(\/.*)?$/.exec(specifier);
77-
const packageDir = match?.[1] ? packageDirs.get(match[1]) : undefined;
157+
const match = WORKSPACE_SPECIFIER.exec(specifier);
158+
const packageName = match?.[1];
159+
const packageDir = packageName ? packageDirs.get(packageName) : undefined;
78160
if (!packageDir) return null;
79-
const manifest = JSON.parse(fs.readFileSync(path.join(packageDir, 'package.json'), 'utf8')) as {
80-
exports?: Record<string, { default?: string; types?: string } | string>;
81-
};
82-
const target = manifest.exports?.[`.${match?.[2] ?? ''}`];
83-
const relativeTarget = typeof target === 'string' ? target : (target?.default ?? target?.types);
84-
if (!relativeTarget) return null;
85-
const resolved = path.resolve(packageDir, relativeTarget);
161+
const target = readExportTarget(packageDir, `.${match?.[2] ?? ''}`);
162+
if (!target) return null;
163+
const resolved = path.resolve(packageDir, target);
86164
return fs.existsSync(resolved) ? resolved : null;
87165
}
88166

@@ -95,7 +173,7 @@ function eagerClosureOfCli(): string[] {
95173
const current = queue.pop();
96174
if (!current || visited.has(current)) continue;
97175
visited.add(current);
98-
for (const specifier of evaluatedModuleRefs(current, fs.readFileSync(current, 'utf8'))) {
176+
for (const specifier of eagerlyEvaluatedModules(current, fs.readFileSync(current, 'utf8'))) {
99177
const resolved = specifier.startsWith('.')
100178
? resolveRelative(current, specifier)
101179
: resolveWorkspace(specifier, packageDirs);
@@ -106,41 +184,68 @@ function eagerClosureOfCli(): string[] {
106184
}
107185

108186
test.for([
109-
{ form: 'side-effect import', code: `import 'node:http';`, evaluates: true },
110-
{ form: 'default value import', code: `import http from 'node:http';`, evaluates: true },
111-
{ form: 'namespace import', code: `import * as http from 'node:http';`, evaluates: true },
112-
{ form: 'named value import', code: `import { request } from 'node:http';`, evaluates: true },
113-
{ form: 'value re-export', code: `export { request } from 'node:http';`, evaluates: true },
114-
{ form: 'star re-export', code: `export * from 'node:http';`, evaluates: true },
187+
// --- static forms ---
188+
{ form: 'side-effect import', code: `import 'node:http';`, eager: true },
189+
{ form: 'default value import', code: `import http from 'node:http';`, eager: true },
190+
{ form: 'namespace import', code: `import * as http from 'node:http';`, eager: true },
191+
{ form: 'named value import', code: `import { request } from 'node:http';`, eager: true },
192+
{ form: 'value re-export', code: `export { request } from 'node:http';`, eager: true },
193+
{ form: 'star re-export', code: `export * from 'node:http';`, eager: true },
115194
{
116195
form: 'mixed value + type import',
117196
code: `import http, { type IncomingMessage } from 'node:http';`,
118-
evaluates: true,
119-
},
120-
{
121-
form: 'type-only default import',
122-
code: `import type http from 'node:http';`,
123-
evaluates: false,
197+
eager: true,
124198
},
199+
{ form: 'type-only default import', code: `import type http from 'node:http';`, eager: false },
125200
{
126201
form: 'type-only named import',
127202
code: `import { type IncomingMessage } from 'node:http';`,
128-
evaluates: false,
203+
eager: false,
129204
},
130205
{
131206
form: 'type-only re-export',
132207
code: `export type { IncomingMessage } from 'node:http';`,
133-
evaluates: false,
208+
eager: false,
209+
},
210+
// --- dynamic import: scope decides, not syntax ---
211+
{ form: 'top-level await import', code: `const m = await import('node:http');`, eager: true },
212+
{ form: 'top-level import().then', code: `import('node:http').then((m) => m);`, eager: true },
213+
{
214+
form: 'top-level immediately-invoked arrow',
215+
code: `(async () => { await import('node:http'); })();`,
216+
eager: true,
217+
},
218+
{
219+
form: 'function-declaration-local import',
220+
code: `async function load() { return await import('node:http'); }`,
221+
eager: false,
222+
},
223+
{
224+
form: 'arrow-local import',
225+
code: `const load = async () => await import('node:http');`,
226+
eager: false,
227+
},
228+
{
229+
form: 'method-local import',
230+
code: `class K { async load() { await import('node:http'); } }`,
231+
eager: false,
232+
},
233+
{
234+
form: 'ternary inside a function (the shipped lazy-load shape)',
235+
code: `async function load(s: boolean) {
236+
return s ? (await import('node:https')).default : (await import('node:http')).default;
237+
}`,
238+
eager: false,
134239
},
135-
{ form: 'dynamic import', code: `const m = await import('node:http');`, evaluates: false },
136-
])('$form is classified as evaluates=$evaluates', ({ code, evaluates }) => {
137-
expect(evaluatedModuleRefs('fixture.ts', code)).toEqual(evaluates ? ['node:http'] : []);
240+
])('$form is eager=$eager', ({ code, eager }) => {
241+
const refs = eagerlyEvaluatedModules('fixture.ts', code);
242+
expect(refs.includes('node:http') || refs.includes('node:https')).toBe(eager);
138243
});
139244

140245
test('the CLI startup import closure never evaluates node:http or node:https', () => {
141246
const offenders: string[] = [];
142247
for (const file of eagerClosureOfCli()) {
143-
for (const specifier of evaluatedModuleRefs(file, fs.readFileSync(file, 'utf8'))) {
248+
for (const specifier of eagerlyEvaluatedModules(file, fs.readFileSync(file, 'utf8'))) {
144249
if (LAZY_HTTP_MODULES.has(specifier)) {
145250
offenders.push(`${path.relative(srcRoot, file)} -> ${specifier}`);
146251
}
@@ -156,7 +261,7 @@ test('the CLI startup import closure never evaluates node:http or node:https', (
156261

157262
test('the CLI startup import closure is reachable and crosses the package boundary', () => {
158263
// Guards the test above from silently passing because the walk found nothing:
159-
// a resolver that returns null for everything would leave both the src side
264+
// a resolver that returned null for everything would leave both the src side
160265
// and the workspace side of the closure empty while the guard stayed green.
161266
const closure = eagerClosureOfCli();
162267
expect(closure.length).toBeGreaterThan(50);

‎src/daemon/client/daemon-client-transport.ts‎

Lines changed: 3 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
import type { RequestProgressSink } from '@agent-device/contracts/progress';
22
import net from 'node:net';
33
import { AppError } from '@agent-device/kernel/errors';
4-
import { readNodeHttpResponseBody } from '../../utils/node-http.ts';
4+
import { loadNodeHttpRequester, readNodeHttpResponseBody } from '../../utils/node-http.ts';
55
import type { DaemonRequest, DaemonResponse } from '../types.ts';
66
import { emitDiagnostic } from '../../utils/diagnostics.ts';
77
import type { DaemonPaths, DaemonTransportPreference } from '../config.ts';
@@ -18,22 +18,6 @@ import { DAEMON_RPC_PROTOCOL_VERSION } from '../http-health.ts';
1818
import { readVersion } from '../../utils/version.ts';
1919

2020
type ResolvedDaemonTransport = 'socket' | 'http';
21-
type NodeHttpRequester = Pick<typeof import('node:http'), 'request'>;
22-
23-
/**
24-
* `node:http` eagerly initializes undici, which costs ~9ms of startup in a
25-
* fresh process. The default local transport is a plain `node:net` socket and
26-
* never issues an HTTP request, so both HTTP modules are loaded on demand —
27-
* only the HTTP transport, remote daemons, and HTTP health checks pay for them.
28-
*/
29-
async function loadHttpRequester(protocol: string): Promise<NodeHttpRequester> {
30-
// `.default` (not the namespace) so this stays the very same module object a
31-
// static `import http from 'node:http'` yields — its `request` property is
32-
// mutable, which the transport's tests rely on to stub outbound requests.
33-
return protocol === 'https:'
34-
? (await import('node:https')).default
35-
: (await import('node:http')).default;
36-
}
3721
type SendRequestOptions = {
3822
onProgress?: RequestProgressSink;
3923
};
@@ -124,7 +108,7 @@ async function readDaemonHttpHealth(info: DaemonInfo): Promise<RemoteDaemonHealt
124108
: null;
125109
if (!endpoint) return { reachable: false };
126110
const url = new URL(endpoint);
127-
const transport = await loadHttpRequester(url.protocol);
111+
const transport = await loadNodeHttpRequester(url.protocol);
128112
const timeoutMs = info.baseUrl
129113
? REMOTE_DAEMON_HEALTHCHECK_TIMEOUT_MS
130114
: LOCAL_DAEMON_HEALTHCHECK_TIMEOUT_MS;
@@ -394,7 +378,7 @@ async function sendHttpRequest(
394378
if (info.baseUrl) {
395379
Object.assign(headers, buildDaemonHttpAuthHeaders(info.token));
396380
}
397-
const transport = await loadHttpRequester(rpcUrl.protocol);
381+
const transport = await loadNodeHttpRequester(rpcUrl.protocol);
398382

399383
return await new Promise((resolve, reject) => {
400384
const request = transport.request(

‎src/remote/daemon-artifacts.ts‎

Lines changed: 4 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ import fs from 'node:fs';
22
import path from 'node:path';
33
import { pipeline } from 'node:stream/promises';
44
import { AppError } from '@agent-device/kernel/errors';
5+
import { loadNodeHttpRequester } from '../utils/node-http.ts';
56
import type { DaemonArtifact, DaemonRequest, DaemonResponse } from '../daemon/types.ts';
67
import {
78
buildDaemonHttpAuthHeaders,
@@ -347,13 +348,9 @@ type DownloadRemoteArtifactParams = {
347348

348349
export async function downloadRemoteArtifact(params: DownloadRemoteArtifactParams): Promise<void> {
349350
const artifactUrl = new URL(buildDaemonArtifactUrl(params.baseUrl, params.artifactId));
350-
// Loaded on demand: `prepareRemoteRequestArtifacts` runs on every CLI
351-
// request, but only a remote daemon ever downloads an artifact, and
352-
// importing `node:http` for a value costs ~9ms of undici init.
353-
const transport =
354-
artifactUrl.protocol === 'https:'
355-
? (await import('node:https')).default
356-
: (await import('node:http')).default;
351+
// `prepareRemoteRequestArtifacts` runs on every CLI request, but only a
352+
// remote daemon ever downloads an artifact, so the HTTP stack loads here.
353+
const transport = await loadNodeHttpRequester(artifactUrl.protocol);
357354
await fs.promises.mkdir(path.dirname(params.destinationPath), { recursive: true });
358355
await new Promise<void>((resolve, reject) => {
359356
let settled = false;

‎src/remote/upload-stream.ts‎

Lines changed: 5 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ import type { IncomingHttpHeaders } from 'node:http';
33
import path from 'node:path';
44
import { pipeline } from 'node:stream/promises';
55
import { AppError } from '@agent-device/kernel/errors';
6-
import { readNodeHttpResponseBody } from '../utils/node-http.ts';
6+
import { loadNodeHttpRequester, readNodeHttpResponseBody } from '../utils/node-http.ts';
77
import {
88
createUploadProgressTransform,
99
type UploadProgressSink,
@@ -64,13 +64,10 @@ async function streamFileToHttpRequestAttempt(options: {
6464
startOffset: number;
6565
progress?: UploadStreamProgressOptions;
6666
}): Promise<UploadStreamResponse> {
67-
// Loaded on demand: this module sits in the CLI's eager import closure via
68-
// the remote-artifact upload client, but only a remote daemon ever uploads.
69-
// Importing `node:http` for a value costs ~9ms of undici init per process.
70-
const transport =
71-
options.url.protocol === 'https:'
72-
? (await import('node:https')).default
73-
: (await import('node:http')).default;
67+
// This module sits in the CLI's eager import closure via the remote-artifact
68+
// upload client, but only a remote daemon ever uploads, so the HTTP stack
69+
// loads here rather than at import time.
70+
const transport = await loadNodeHttpRequester(options.url.protocol);
7471
const payloadSize = fs.statSync(options.payloadPath).size;
7572
const headers = buildUploadRequestHeaders(options.headers, options.startOffset, payloadSize);
7673
emitUploadAttemptStarted(options.progress, options.startOffset, payloadSize);

‎src/utils/node-http.ts‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,31 @@
11
import type { IncomingMessage } from 'node:http';
22
import { AppError } from '@agent-device/kernel/errors';
33

4+
/** The slice of `node:http` / `node:https` an outbound request needs. */
5+
export type NodeHttpRequester = Pick<typeof import('node:http'), 'request'>;
6+
7+
/**
8+
* Loads `node:http` / `node:https` on demand, at the call site that actually
9+
* issues a request.
10+
*
11+
* Importing either for a VALUE initializes undici and, under
12+
* `NODE_USE_SYSTEM_CA=1`, the platform trust store -- ~79ms per process. Several
13+
* modules that only *might* speak HTTP (the daemon transport, remote artifact
14+
* upload/download) sit in the CLI's eager import closure, where the default
15+
* daemon transport is a plain `node:net` socket and no HTTP request is ever
16+
* made. `src/__tests__/cli-startup-import-closure.test.ts` fails the build if a
17+
* static import of either module reappears in that closure.
18+
*
19+
* Resolves `.default`, not the namespace, so callers get the very same module
20+
* object a static `import http from 'node:http'` yields -- its `request`
21+
* property is mutable, which the transport tests rely on to stub requests.
22+
*/
23+
export async function loadNodeHttpRequester(protocol: string): Promise<NodeHttpRequester> {
24+
return protocol === 'https:'
25+
? (await import('node:https')).default
26+
: (await import('node:http')).default;
27+
}
28+
429
export function readNodeHttpResponseBody(res: IncomingMessage): Promise<string> {
530
return new Promise((resolve, reject) => {
631
let body = '';

0 commit comments

Comments
 (0)