Skip to content

Commit e3bc3df

Browse files
ci(publish): fail the release if a published package cannot be imported
Two releases have now shipped artifacts that no consumer could import: salesforce#35 dist/ still referenced the internal @agentscript/* scope after the publish rewrite, so consumers hit ERR_MODULE_NOT_FOUND. salesforce#71 agentforce, compiler and agentforce-dialect were published pinning @sf-agentscript/[email protected] while their dist/ had been compiled against a newer language, so consumers hit a SyntaxError for the missing export nullLiteralValidationPass and a TypeError for the missing variantMatch builder. Neither is visible from inside the workspace: pnpm resolves siblings from source, so the packed manifest is never exercised, and packing from the workspace always produces a self-consistent set of tarballs. The failure only appears once a dependent resolves a sibling from the registry, i.e. only in a real consumer install. This adds that install as a publish step. scripts/verify-published-packages.mjs installs each publishable package from the registry into a clean directory and imports its public entry point in a child process, skipping entries that are not bare-importable by design (lsp-server, lsp-browser, monaco, parser-tree-sitter). publish.mjs runs it after `changeset publish`, so a broken release fails the workflow in minutes instead of being found by the first user to install it. Verified against the currently published packages: it flags [email protected], [email protected] and [email protected] with their exact errors, and passes [email protected], [email protected] and [email protected]. Refs salesforce#71
1 parent 2109cb9 commit e3bc3df

2 files changed

Lines changed: 264 additions & 0 deletions

File tree

‎scripts/publish.mjs‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -136,3 +136,16 @@ execFileSync(
136136
stdio: 'inherit',
137137
}
138138
);
139+
140+
// Step 6: Verify what we just published is actually installable and importable.
141+
// The workspace can't tell us this: pnpm resolves siblings from source, so a
142+
// stale dependency pin or an unrewritten specifier only breaks once a consumer
143+
// resolves from the registry (see #35, #71 and #73). Fails the job if a release
144+
// is broken, so it is caught in minutes rather than by the first user to try
145+
// it. Because `pnpm -r publish` is idempotent, this also re-checks registry
146+
// state on no-op re-runs — a broken earlier release keeps failing the sync
147+
// until it is fixed.
148+
execFileSync('node', [join(ROOT, 'scripts', 'verify-published-packages.mjs')], {
149+
cwd: ROOT,
150+
stdio: 'inherit',
151+
});
Lines changed: 251 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,251 @@
1+
/*
2+
* Copyright (c) 2026, Salesforce, Inc.
3+
* All rights reserved.
4+
* SPDX-License-Identifier: Apache-2.0
5+
* For full license text, see the LICENSE file in the repo root or https://www.apache.org/licenses/LICENSE-2.0
6+
*/
7+
8+
/**
9+
* Post-publish smoke test: install what we just published straight from the
10+
* registry, the way a consumer does, and import it.
11+
*
12+
* Two releases have now shipped artifacts that could not be imported at all:
13+
*
14+
* #35 - dist/ still referenced the internal @agentscript/* scope after the
15+
* publish rewrite, so consumers hit ERR_MODULE_NOT_FOUND.
16+
* #71 - agentforce, compiler and agentforce-dialect were published pinning
17+
* @sf-agentscript/[email protected] while their dist/ had been compiled
18+
* against a newer language, so consumers hit
19+
* SyntaxError: ... does not provide an export named
20+
* 'nullLiteralValidationPass'
21+
* and
22+
* TypeError: ....variantMatch is not a function
23+
*
24+
* Why this runs *after* the publish step rather than in CI: inside the
25+
* workspace pnpm resolves siblings from source, and packing from the workspace
26+
* always produces a self-consistent set of tarballs. Both bugs only surface once
27+
* a dependent resolves a sibling from the *registry* - i.e. only in a real
28+
* consumer install. A pre-publish pack-and-install check would happily pass
29+
* while the published artifact is broken, so we verify the real thing.
30+
*
31+
* Note: this deliberately uses `npm`, not `pnpm`, for the consumer install -
32+
* pnpm-workspace.yaml sets `minimumReleaseAge: 1440`, which would refuse to
33+
* install a package published seconds ago.
34+
*
35+
* Usage:
36+
* node scripts/verify-published-packages.mjs
37+
* verify every publishable workspace package at its current version
38+
* node scripts/verify-published-packages.mjs @sf-agentscript/[email protected]
39+
* verify explicit <name>@<version> specs (handy for spot-checking npm)
40+
*
41+
* Exits non-zero if any published package cannot be imported, so a broken
42+
* release fails the workflow loudly instead of sitting silently on npm.
43+
*/
44+
45+
import { execFileSync } from 'node:child_process';
46+
import { mkdtempSync, writeFileSync, readFileSync, existsSync } from 'node:fs';
47+
import { tmpdir } from 'node:os';
48+
import { join } from 'node:path';
49+
50+
const ROOT = new URL('..', import.meta.url).pathname;
51+
52+
/**
53+
* Run a package-manager binary. On Windows `npm`/`pnpm` are `.cmd` shims, and
54+
* since Node 20 execFile refuses to spawn those without a shell (EINVAL), so we
55+
* opt into one there. CI runs on Linux, where the plain binary is spawned.
56+
*/
57+
const run = (bin, args, opts = {}) =>
58+
execFileSync(bin, args, {
59+
encoding: 'utf8',
60+
shell: process.platform === 'win32',
61+
...opts,
62+
});
63+
64+
// How long to wait for a freshly published version to reach the registry.
65+
const REGISTRY_RETRIES = 5;
66+
const REGISTRY_RETRY_DELAY_MS = 5000;
67+
const IMPORT_TIMEOUT_MS = 60_000;
68+
69+
/**
70+
* Entry points that are not meant to be imported bare in Node. Keyed by the
71+
* package name without its scope, so this works either side of the publish-time
72+
* scope rewrite. Each reason was confirmed by importing the published package.
73+
*/
74+
const NOT_BARE_IMPORTABLE = {
75+
'lsp-server':
76+
'server entry - a bare import throws "Connection input stream is not set" by design',
77+
'lsp-browser': 'browser bundle - calls browser-only APIs on import',
78+
monaco: 'ships TypeScript source as its entry (src/index.ts)',
79+
'parser-tree-sitter':
80+
'native binding - needs a platform-aware check, not a bare import',
81+
};
82+
83+
const readJson = path => JSON.parse(readFileSync(path, 'utf8'));
84+
85+
/** The specifier a consumer actually loads for `import '<pkg>'`. */
86+
function defaultEntry(pkgJson) {
87+
const dot = pkgJson.exports?.['.'];
88+
const target =
89+
typeof dot === 'string' ? dot : (dot?.import ?? dot?.default ?? null);
90+
return target ?? pkgJson.main ?? null;
91+
}
92+
93+
const unscoped = name =>
94+
name.includes('/') ? name.slice(name.indexOf('/') + 1) : name;
95+
96+
/** Why this package is not import-tested, or null if it should be. */
97+
function skipReason(pkgJson, pkgPath) {
98+
if (pkgJson.private) return 'private';
99+
100+
const declared = NOT_BARE_IMPORTABLE[unscoped(pkgJson.name)];
101+
if (declared) return declared;
102+
103+
if (existsSync(join(pkgPath, 'binding.gyp'))) {
104+
return 'native addon (binding.gyp)';
105+
}
106+
107+
const entry = defaultEntry(pkgJson);
108+
if (!entry) return 'no entry point declared';
109+
if (!/\.(js|mjs|cjs)$/.test(entry)) {
110+
return `entry is not JavaScript (${entry})`;
111+
}
112+
return null;
113+
}
114+
115+
const sleep = ms =>
116+
execFileSync(process.execPath, [
117+
'-e',
118+
`Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, ${ms})`,
119+
]);
120+
121+
/** True once <name>@<version> is resolvable on the registry. */
122+
function onRegistry(name, version) {
123+
for (let attempt = 1; attempt <= REGISTRY_RETRIES; attempt++) {
124+
try {
125+
const out = run('npm', ['view', `${name}@${version}`, 'version'], {
126+
stdio: ['ignore', 'pipe', 'ignore'],
127+
});
128+
if (out.trim()) return true;
129+
} catch (e) {
130+
// A missing npm binary must not be mistaken for a missing package.
131+
if (e.code === 'ENOENT' || e.code === 'EINVAL') {
132+
throw new Error('could not run "npm" - is npm on PATH?');
133+
}
134+
// Otherwise: not on the registry yet.
135+
}
136+
if (attempt < REGISTRY_RETRIES) sleep(REGISTRY_RETRY_DELAY_MS);
137+
}
138+
return false;
139+
}
140+
141+
const firstErrorLine = output => {
142+
const lines = output.toString().split('\n');
143+
const hit = lines.map(l => l.trim()).find(l => /error/i.test(l));
144+
return (hit ?? lines[0] ?? '').slice(0, 200);
145+
};
146+
147+
/**
148+
* Install <name>@<version> into a clean directory exactly as a consumer would,
149+
* then import it in a child process. Returns an error string, or null on success.
150+
*/
151+
function verifyConsumerInstall(name, version) {
152+
const dir = mkdtempSync(join(tmpdir(), 'agentscript-smoke-'));
153+
writeFileSync(
154+
join(dir, 'package.json'),
155+
JSON.stringify({
156+
name: 'agentscript-smoke-test',
157+
version: '0.0.0',
158+
private: true,
159+
type: 'module',
160+
})
161+
);
162+
163+
try {
164+
run('npm', ['install', `${name}@${version}`, '--no-audit', '--no-fund'], {
165+
cwd: dir,
166+
stdio: ['ignore', 'pipe', 'pipe'],
167+
});
168+
} catch (e) {
169+
return `install failed: ${firstErrorLine(e.stderr || e.message)}`;
170+
}
171+
172+
try {
173+
execFileSync(
174+
process.execPath,
175+
['--input-type=module', '-e', `await import(${JSON.stringify(name)});`],
176+
{
177+
cwd: dir,
178+
encoding: 'utf8',
179+
timeout: IMPORT_TIMEOUT_MS,
180+
stdio: ['ignore', 'pipe', 'pipe'],
181+
}
182+
);
183+
} catch (e) {
184+
return `import failed: ${firstErrorLine(e.stderr || e.message)}`;
185+
}
186+
187+
return null;
188+
}
189+
190+
/** Explicit `<name>@<version>` specs, or the publishable workspace packages. */
191+
function targets() {
192+
const args = process.argv.slice(2);
193+
if (args.length > 0) {
194+
return args.map(spec => {
195+
const at = spec.lastIndexOf('@');
196+
if (at <= 0) throw new Error(`expected <name>@<version>, got "${spec}"`);
197+
return { name: spec.slice(0, at), version: spec.slice(at + 1) };
198+
});
199+
}
200+
201+
const workspace = JSON.parse(
202+
run('pnpm', ['-r', 'list', '--json', '--depth', '-1'], { cwd: ROOT })
203+
);
204+
205+
const out = [];
206+
for (const pkg of workspace) {
207+
const pkgJsonPath = join(pkg.path, 'package.json');
208+
if (!existsSync(pkgJsonPath)) continue;
209+
const pkgJson = readJson(pkgJsonPath);
210+
const skip = skipReason(pkgJson, pkg.path);
211+
if (skip) {
212+
console.log(` - ${pkgJson.name} - skipped (${skip})`);
213+
continue;
214+
}
215+
out.push({ name: pkgJson.name, version: pkgJson.version });
216+
}
217+
return out;
218+
}
219+
220+
// ---------------------------------------------------------------------------
221+
222+
console.log('\nVerifying published packages install and import cleanly\n');
223+
224+
const failures = [];
225+
let checked = 0;
226+
227+
for (const { name, version } of targets()) {
228+
if (!onRegistry(name, version)) {
229+
console.log(` - ${name}@${version} - skipped (not on the registry)`);
230+
continue;
231+
}
232+
233+
const error = verifyConsumerInstall(name, version);
234+
checked++;
235+
if (error) {
236+
failures.push({ name, version, error });
237+
console.log(` FAIL ${name}@${version}\n ${error}`);
238+
} else {
239+
console.log(` ok ${name}@${version}`);
240+
}
241+
}
242+
243+
if (failures.length > 0) {
244+
console.error(
245+
`\n${failures.length} of ${checked} published package(s) cannot be imported by a consumer.`
246+
);
247+
console.error('The release is on npm but broken - publish a fix now.\n');
248+
process.exit(1);
249+
}
250+
251+
console.log(`\nAll ${checked} published package(s) import cleanly.\n`);

0 commit comments

Comments
 (0)