Skip to content

Commit 4382322

Browse files
cliffhallclaude
andcommitted
ci: also hard-fail the async half of the crash class (fourth review)
page.on("pageerror") fires for uncaught SYNCHRONOUS exceptions only. The async twin of the #1612 shape — the same empty-stub TypeError reached through an await/.then(), or a failed dynamic import (this app lazy-loads chunks) — lands on the console channel as `Uncaught (in promise) …` / `Failed to fetch dynamically imported module`, which were diagnostic-only, so the smoke would have passed. Hard-fail console errors matching /^Uncaught\b|Failed to fetch dynamically imported module/; every other console error stays a printed non-fatal note. The prefixes are unambiguous (a font/CDN miss reads "Failed to load resource: net::ERR_…"; React warnings never start with "Uncaught"), so this can't reintroduce the flake the earlier round fixed. Verified: an unhandled rejection at module init now fails the smoke where it previously wouldn't. Also prod-scope the AGENTS.md note that the "externalized" string isn't a runtime message (under `npm run dev` Vite's stub is a Proxy that console.warns it), and document both the sync and async failure signals. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5
1 parent b97b407 commit 4382322

2 files changed

Lines changed: 37 additions & 13 deletions

File tree

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -219,7 +219,7 @@ gh project item-edit --project-id PVT_kwDOCt2Azc4BJVxt --id "$ITEM_ID" --field-i
219219
- **`smoke` is NOT part of `validate`** — it is included in `npm run ci`. It runs `smoke:launcher` (`--help` dispatch) plus the prod `smoke:cli` / `smoke:tui` / `smoke:web` / `smoke:web:browser`, and contains **no build commands** — it assumes the cli/tui/launcher bundles already exist (a full `validate` builds them; `smoke:web` builds `clients/web/dist` on demand). CI runs `validate`, then the `coverage` gate (which also covers the web integration project), then `smoke` (with Playwright chromium installed just before it, since `smoke:web:browser` needs it). Storybook is the only CI step left out (see below).
220220
- `smoke:launcher` (`scripts/smoke-launcher.mjs`) runs the built launcher with `--help`, `--cli --help`, and `--tui --help`, asserting each exits 0 and prints that mode's usage banner (which also proves the launcher resolved and loaded the right client build). It's the cheap dispatch check before the heavier prod smokes below.
221221
- `smoke:web` (`scripts/smoke-web.mjs`) starts `mcp-inspector --web` (prod, no `--dev`) against the built `clients/web/dist` and asserts `GET /` serves the SPA (HTTP 200) with the injected `__INSPECTOR_API_TOKEN__`. Prod `--web` serves from `clients/web/dist`, which ships in the published package but is absent in a fresh checkout — the runner builds it on demand (`build:client` = `vite build`) on first launch, or exits with an actionable error if that build can't run (see `clients/web/server/ensure-web-build.ts` and the launcher README). `--dev` runs Vite directly and never needs `dist`. It shares the spawn/readiness/teardown helper (`scripts/lib/prod-web-server.mjs`) with `smoke:web:browser`, so the two can't drift.
222-
- `smoke:web:browser` (`scripts/smoke-web-browser.mjs`, #1615) goes a step further than `smoke:web`: it boots the same prod `--web` server and then actually **runs** the bundle in headless Chromium (Playwright — already a `clients/web` devDependency for the Storybook tests), asserting the app renders its first meaningful frame (the "Add Servers" control) with **no uncaught page error**. `smoke:web` only checks the served HTML, so a Node built-in reaching the browser bundle slipped through it; this smoke catches that regression as a *class* (e.g. #1612). The mechanism is the uncaught error, not a magic string: under Vite the excluded module becomes an empty stub and the first *call* into it (e.g. `fs.readFileSync(...)` during a transitive module's init) throws a `TypeError` that aborts app mount — a `pageerror` the smoke fails on. The literal `Module "…" has been externalized` text is a **build-time** warning (`vite build` / `npm run build`), not a runtime message, so the browser never sees it; and an externalized import that is never *called* ships a harmless `{}` and is invisible here by design. Console errors are printed as diagnostics, not failures (so a benign font-CDN or React-warning `console.error` doesn't flake CI). Playwright is resolved via `createRequire` based at `clients/web/package.json` — a bare `import("playwright")` would resolve relative to `scripts/`, not the cwd, so it can't be reached that way (it only appears to work when an ancestor `node_modules` carries playwright, and fails in CI, which has none). The npm script's `cd clients/web` exists only so `npx playwright install chromium` finds the local playwright bin (a no-op when already installed).
222+
- `smoke:web:browser` (`scripts/smoke-web-browser.mjs`, #1615) goes a step further than `smoke:web`: it boots the same prod `--web` server and then actually **runs** the bundle in headless Chromium (Playwright — already a `clients/web` devDependency for the Storybook tests), asserting the app renders its first meaningful frame (the "Add Servers" control) with **no uncaught error**. `smoke:web` only checks the served HTML, so a Node built-in reaching the browser bundle slipped through it; this smoke catches that regression as a *class* (e.g. #1612). The mechanism is the uncaught error, not a magic string: under Vite the excluded module becomes an empty stub and the first *call* into it (e.g. `fs.readFileSync(...)` during a transitive module's init) throws a `TypeError` that aborts app mount. A *synchronous* such throw fires `pageerror`; its *async* twin (the same `TypeError` via `await`/`.then()`, or a failed dynamic import) is logged on the console channel as `Uncaught (in promise) …` / `Failed to fetch dynamically imported module` — the smoke hard-fails on both. The literal `Module "…" has been externalized` text is, **in a prod build**, a build-time warning (`vite build` / `npm run build`), not a runtime message, so the browser never sees it (under `npm run dev` Vite's stub is instead a `Proxy` that `console.warn`s that string at runtime); and an externalized import that is never *called* ships a harmless `{}` and is invisible here by design. Every *other* console error is printed as a diagnostic, not a failure (so a benign font-CDN or React-warning `console.error` doesn't flake CI). Playwright is resolved via `createRequire` based at `clients/web/package.json` — a bare `import("playwright")` would resolve relative to `scripts/`, not the cwd, so it can't be reached that way (it only appears to work when an ancestor `node_modules` carries playwright, and fails in CI, which has none). The npm script's `cd clients/web` exists only so `npx playwright install chromium` finds the local playwright bin (a no-op when already installed).
223223
- `smoke:cli` (`scripts/smoke-cli.mjs`) drives `mcp-inspector --cli` through the built launcher against the bundled stdio test server via a temp `--catalog`: it asserts `tools/list` returns the server's tools (real connect over stdio), the default writable catalog is seeded empty on first run, a missing read-only `--config` errors without seeding, and `--catalog` + `--config` is rejected. `smoke:tui` (`scripts/smoke-tui.mjs`) launches `mcp-inspector --tui --catalog <temp>` and asserts the Ink app renders its first frame (the "MCP Servers" panel) within a timeout, then SIGTERMs it — a shallow boot/render check, not full interaction. **`smoke:tui` is local-only: it self-skips when `process.env.CI` is set**, because the Ink TUI needs a real TTY (raw mode) that headless CI lacks — so run it (via `npm run smoke`) on your own machine before pushing. Both build `test-servers/build` on demand if it's missing.
224224
- Storybook play-function tests (`clients/web` `test:storybook`) run in headless Chromium via `@vitest/browser-playwright` (~10s). They are part of `npm run ci` (which installs Playwright chromium first); kept out of `validate` because they need the browser binary and are slower than the unit suite.
225225

‎scripts/smoke-web-browser.mjs‎

Lines changed: 36 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -26,10 +26,21 @@
2626
* import that is never *called* ships a harmless empty object and is invisible
2727
* to this smoke by design — an unused Node import doesn't crash the app.)
2828
*
29-
* `console.error` is NOT a hard failure: the console is where Chromium also
30-
* reports benign things a boot smoke shouldn't fail on — a failed subresource
31-
* load (e.g. the Google-Fonts `<link>` in index.html on a network-restricted
32-
* box) or a React key/prop warning. Console errors are printed as diagnostics.
29+
* Two channels carry the failure. A *synchronous* uncaught exception (the
30+
* CASE-1 shape above — a stub call during module init) fires `pageerror`. Its
31+
* *async* twin — the same `TypeError` reached through an `await`/`.then()`, or a
32+
* failed dynamic import (this app lazy-loads chunks) — is NOT a `pageerror`;
33+
* Chromium logs it on the **console** channel as `Uncaught (in promise) …` /
34+
* `Failed to fetch dynamically imported module`. Both are hard failures.
35+
*
36+
* Every *other* `console.error` is NOT a hard failure: the console is where
37+
* Chromium also reports benign things a boot smoke shouldn't fail on — a failed
38+
* subresource load (e.g. the Google-Fonts `<link>` in index.html on a
39+
* network-restricted box) or a React key/prop warning. Those are printed as
40+
* diagnostics. The `Uncaught` / dynamic-import prefixes are unambiguous — a
41+
* font/CDN miss reads `Failed to load resource: net::ERR_…` and a React warning
42+
* never starts with `Uncaught` — so hard-failing on them can't reintroduce that
43+
* flake.
3344
*
3445
* Playwright lives in clients/web's node_modules, so it's resolved with a
3546
* `createRequire` based at clients/web/package.json rather than a bare
@@ -61,6 +72,12 @@ const HOST = "127.0.0.1";
6172
const PORT = process.env.SMOKE_WEB_BROWSER_PORT ?? "6298";
6273
const TOKEN = "smoke-web-browser-token";
6374

75+
// Console messages that are the async half of the uncaught-crash class (an
76+
// unhandled promise rejection or a failed dynamic import). Hard failures, unlike
77+
// benign console noise. See the header comment for why this can't reintroduce
78+
// the font/CDN flake.
79+
const FATAL_CONSOLE = /^Uncaught\b|Failed to fetch dynamically imported module/;
80+
6481
const server = startProdWebServer({ host: HOST, port: PORT, token: TOKEN });
6582
let browser = null;
6683

@@ -105,11 +122,12 @@ try {
105122
browser = await loadChromium();
106123
const page = await browser.newPage();
107124

108-
// Uncaught page errors are the hard failure — a Node-only module reaching the
109-
// browser bundle surfaces here as a TypeError when its empty stub is called.
125+
// Uncaught (synchronous) page errors are a hard failure — a Node-only module
126+
// reaching the browser bundle surfaces here as a TypeError when its empty stub
127+
// is called during module init.
110128
const pageErrors = [];
111-
// Console errors are diagnostic only (see the header comment for why they're
112-
// not a blanket failure).
129+
// Console errors are diagnostic only EXCEPT those matching FATAL_CONSOLE (the
130+
// async half of the same crash class) — see the header comment.
113131
const consoleErrors = [];
114132
page.on("pageerror", (err) =>
115133
pageErrors.push(err instanceof Error ? err.message : String(err)),
@@ -161,15 +179,21 @@ try {
161179
);
162180
}
163181

164-
if (pageErrors.length > 0) {
165-
await fail(`app logged uncaught page error(s): ${pageErrors.join("; ")}`);
182+
// Hard failures: any uncaught (sync) page error, plus console errors that are
183+
// the async half of the class (unhandled rejection / failed dynamic import).
184+
const fatalConsole = consoleErrors.filter((m) => FATAL_CONSOLE.test(m));
185+
if (pageErrors.length > 0 || fatalConsole.length > 0) {
186+
await fail(
187+
`app logged uncaught error(s): ${[...pageErrors, ...fatalConsole].join("; ")}`,
188+
);
166189
}
167190

168191
// Non-fatal console errors: surface them so a real problem isn't invisible,
169192
// without failing the smoke on benign subresource/warning noise.
170-
if (consoleErrors.length > 0) {
193+
const benignConsole = consoleErrors.filter((m) => !FATAL_CONSOLE.test(m));
194+
if (benignConsole.length > 0) {
171195
console.log(
172-
`smoke:web:browser note — ${consoleErrors.length} non-fatal console error(s): ${consoleErrors.join("; ")}`,
196+
`smoke:web:browser note — ${benignConsole.length} non-fatal console error(s): ${benignConsole.join("; ")}`,
173197
);
174198
}
175199

0 commit comments

Comments
 (0)