Skip to content

Commit 8e7452a

Browse files
maxpuppetmaxpuppet
committed
fix(plugins): mirror core root-package deps used by core dist code
Extend MIRRORED_CORE_RUNTIME_DEP_NAMES from ["semver", "tslog"] to also include @agentclientprotocol/sdk, @lydell/node-pty, croner, dotenv, jiti, json5, jszip, markdown-it, tar, and web-push. These are all declared as direct dependencies in the openclaw root package.json and imported by core source code (src/acp/*, src/cron/*, src/config/*, src/infra/{archive,backup,dotenv,push-web}.ts, src/markdown/ir.ts, src/plugin-sdk/root-alias.cjs, src/plugins/jiti-loader-cache.ts, src/process/supervisor/adapters/pty.ts, etc), but the existing collectMirroredPackageRuntimeDeps allowlist only covered semver and tslog. The dynamic collectRootDistMirroredRuntimeDeps scan does pick up imports that have an extension package.json owner (for example memory-core declares chokidar, matrix declares jiti and markdown-it). For deps with no extension owner, or for setups where the owning extension is not enabled, those imports never make it into the runtime-deps mirror and Node fails to resolve them at runtime, e.g.: Cannot find package 'chokidar' imported from .../plugin-runtime-deps/openclaw-<ver>/dist/qmd-manager-...js Also add a static drift guard test that walks src/ for value imports of root-package runtime deps and fails when one is neither in MIRRORED_CORE_RUNTIME_DEP_NAMES nor declared by any extension's package.json (with an explicit allowlist for known-transitive or build/type-only imports such as chalk, ipaddr.js, file-type, proxy-agent, typescript, qrcode). The guard caught @lydell/node-pty during this change. Refs #74199.
1 parent beb1d9b commit 8e7452a

3 files changed

Lines changed: 237 additions & 1 deletion

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ Docs: https://docs.openclaw.ai
1919
- Browser/gateway: ignore Playwright dialog-close races from `Page.handleJavaScriptDialog` so browser automation no longer crashes the Gateway when a dialog disappears before Playwright accepts it. (#40067) Thanks @randyjtw.
2020
- Cron/Gateway: defer missed isolated agent-turn catch-up out of the channel startup window, so overdue cron work cannot starve Discord or Telegram while providers connect after a restart. Thanks @vincentkoc.
2121
- Plugins/runtime-deps: prune stale `openclaw-unknown-*` bundled runtime dependency roots during Gateway startup while keeping recent or locked roots, so old staging debris cannot keep growing across restarts. Thanks @vincentkoc.
22+
- Plugins/runtime-deps: include ten more root-package runtime dependencies (`@agentclientprotocol/sdk`, `@lydell/node-pty`, `croner`, `dotenv`, `jiti`, `json5`, `jszip`, `markdown-it`, `tar`, `web-push`) in `MIRRORED_CORE_RUNTIME_DEP_NAMES` so they are mirrored into the runtime-deps tree alongside `semver` and `tslog`, preventing `Cannot find package 'X'` failures from core dist code (for example `qmd-manager`, `cron/schedule`, `infra/archive`, `infra/push-web`, `infra/backup-create`, `process/supervisor/adapters/pty`) when no enabled extension owns the dependency. Adds a static drift guard test that scans `src/` for value imports of root-package deps and fails CI when one is missing from the mirror allowlist or extension-owned set. Refs #74199. Thanks @maxpuppet.
2223
- Ollama: compose caller abort signals with guarded-fetch timeouts for native `/api/chat` streams, so `/stop` and early cancellation still interrupt local Ollama requests that also carry provider timeout budgets. Refs #74133. Thanks @obviyus.
2324
- Doctor/TTS: migrate legacy `messages.tts.enabled`, agent TTS, channel TTS, and voice-call plugin TTS toggles to `auto` mode during `openclaw doctor --fix`, matching the documented TTS config contract. Thanks @vincentkoc.
2425
- CLI/logs: fall back to the configured Gateway file log when implicit loopback Gateway connections close or time out before or during `logs.tail`, so `openclaw logs` still works while diagnosing local-model Gateway disconnects. Refs #74078. Thanks @sakalaboator.

‎src/plugins/bundled-runtime-deps.test.ts‎

Lines changed: 222 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ import { spawn, spawnSync } from "node:child_process";
22
import { createHash } from "node:crypto";
33
import { EventEmitter } from "node:events";
44
import fs from "node:fs";
5+
import { Module } from "node:module";
56
import os from "node:os";
67
import path from "node:path";
78
import { afterEach, describe, expect, it, vi } from "vitest";
@@ -2929,3 +2930,224 @@ describe("ensureBundledPluginRuntimeDeps", () => {
29292930
expect(fs.existsSync(path.join(pluginRoot, "node_modules", "zod", "package.json"))).toBe(true);
29302931
});
29312932
});
2933+
2934+
describe("MIRRORED_CORE_RUNTIME_DEP_NAMES drift guard", () => {
2935+
// Intentionally not mirrored at runtime: build-only / type-only / TUI-only
2936+
// tooling and packages that resolve transitively through other mirrored deps.
2937+
// If you change this set, document why in the comment beside the entry.
2938+
const KNOWN_UNMIRRORED_BARE_IMPORTS = new Set<string>([
2939+
"@mariozechner/pi-tui", // TUI mode runs from npm-global, not the gateway runtime mirror
2940+
"chalk", // available transitively via mirrored deps
2941+
"file-type", // available transitively via mirrored deps
2942+
"global-agent", // proxy bootstrap, only loaded when HTTP_PROXY is set
2943+
"ipaddr.js", // available transitively via mirrored deps
2944+
"proxy-agent", // available transitively via mirrored deps
2945+
"qrcode", // type-only import in src/media/qr-runtime.ts
2946+
"typescript", // CLI/dev only (api-baseline, jiti-runtime-api)
2947+
]);
2948+
2949+
function locateRepoRoot(): string {
2950+
let dir = path.resolve(import.meta.dirname);
2951+
for (let depth = 0; depth < 10; depth += 1) {
2952+
const candidate = path.join(dir, "package.json");
2953+
if (fs.existsSync(candidate)) {
2954+
try {
2955+
const data = JSON.parse(fs.readFileSync(candidate, "utf8")) as { name?: string };
2956+
if (data.name === "openclaw") {
2957+
return dir;
2958+
}
2959+
} catch {
2960+
// fall through
2961+
}
2962+
}
2963+
const parent = path.dirname(dir);
2964+
if (parent === dir) {
2965+
break;
2966+
}
2967+
dir = parent;
2968+
}
2969+
throw new Error("could not locate openclaw repo root from test file");
2970+
}
2971+
2972+
function readPackageJsonDeps(packageJsonPath: string): Set<string> {
2973+
const out = new Set<string>();
2974+
if (!fs.existsSync(packageJsonPath)) {
2975+
return out;
2976+
}
2977+
let parsed: {
2978+
dependencies?: Record<string, string>;
2979+
optionalDependencies?: Record<string, string>;
2980+
};
2981+
try {
2982+
parsed = JSON.parse(fs.readFileSync(packageJsonPath, "utf8"));
2983+
} catch {
2984+
return out;
2985+
}
2986+
for (const name of Object.keys(parsed.dependencies ?? {})) {
2987+
out.add(name);
2988+
}
2989+
for (const name of Object.keys(parsed.optionalDependencies ?? {})) {
2990+
out.add(name);
2991+
}
2992+
return out;
2993+
}
2994+
2995+
function collectExtensionOwnedDeps(repoRoot: string): Set<string> {
2996+
const out = new Set<string>();
2997+
const extensionsDir = path.join(repoRoot, "extensions");
2998+
if (!fs.existsSync(extensionsDir)) {
2999+
return out;
3000+
}
3001+
for (const entry of fs.readdirSync(extensionsDir, { withFileTypes: true })) {
3002+
if (!entry.isDirectory()) {
3003+
continue;
3004+
}
3005+
for (const name of readPackageJsonDeps(
3006+
path.join(extensionsDir, entry.name, "package.json"),
3007+
)) {
3008+
out.add(name);
3009+
}
3010+
}
3011+
return out;
3012+
}
3013+
3014+
function walkCoreSourceFiles(repoRoot: string): string[] {
3015+
const srcDir = path.join(repoRoot, "src");
3016+
const files: string[] = [];
3017+
const queue: string[] = [srcDir];
3018+
while (queue.length > 0) {
3019+
const current = queue.shift();
3020+
if (!current) {
3021+
continue;
3022+
}
3023+
for (const entry of fs.readdirSync(current, { withFileTypes: true })) {
3024+
const full = path.join(current, entry.name);
3025+
if (entry.isDirectory()) {
3026+
if (entry.name === "node_modules" || entry.name.startsWith(".")) {
3027+
continue;
3028+
}
3029+
queue.push(full);
3030+
continue;
3031+
}
3032+
if (!entry.isFile()) {
3033+
continue;
3034+
}
3035+
if (
3036+
/\.test\.tsx?$/u.test(entry.name) ||
3037+
/\.e2e\.test\.tsx?$/u.test(entry.name) ||
3038+
/\.test-helpers?\.tsx?$/u.test(entry.name) ||
3039+
/\.test-fixture\.tsx?$/u.test(entry.name) ||
3040+
entry.name.endsWith(".d.ts") ||
3041+
!/\.(?:ts|tsx|cjs|mjs|js)$/u.test(entry.name)
3042+
) {
3043+
continue;
3044+
}
3045+
files.push(full);
3046+
}
3047+
}
3048+
return files;
3049+
}
3050+
3051+
function packageNameFromBareSpecifier(specifier: string): string | null {
3052+
if (
3053+
specifier.startsWith(".") ||
3054+
specifier.startsWith("/") ||
3055+
specifier.startsWith("node:") ||
3056+
specifier.startsWith("#")
3057+
) {
3058+
return null;
3059+
}
3060+
const [first, second] = specifier.split("/");
3061+
if (!first) {
3062+
return null;
3063+
}
3064+
return first.startsWith("@") && second ? `${first}/${second}` : first;
3065+
}
3066+
3067+
// Match value imports (`import x from 'y'`, `import 'y'`, `require('y')`,
3068+
// `import('y')`) but skip `import type` to avoid noise from type-only imports.
3069+
const VALUE_IMPORT_PATTERNS = [
3070+
/(?:^|[;\n])\s*import\s+(?!type\b)(?:[^'"()]+?\s+from\s+)?["']([^"']+)["']/g,
3071+
/\brequire\s*\(\s*["']([^"']+)["']\s*\)/g,
3072+
/\bimport\s*\(\s*["']([^"']+)["']\s*\)/g,
3073+
] as const;
3074+
3075+
it("every value-imported root-package dep in src/ is mirrored or owned by an extension", () => {
3076+
const repoRoot = locateRepoRoot();
3077+
const rootDeps = readPackageJsonDeps(path.join(repoRoot, "package.json"));
3078+
const extensionDeps = collectExtensionOwnedDeps(repoRoot);
3079+
const mirroredCore = new Set<string>([
3080+
"@agentclientprotocol/sdk",
3081+
"@lydell/node-pty",
3082+
"croner",
3083+
"dotenv",
3084+
"jiti",
3085+
"json5",
3086+
"jszip",
3087+
"markdown-it",
3088+
"semver",
3089+
"tar",
3090+
"tslog",
3091+
"web-push",
3092+
]);
3093+
const nodeBuiltins = new Set<string>(Module.builtinModules);
3094+
3095+
const violations = new Map<string, string>();
3096+
for (const file of walkCoreSourceFiles(repoRoot)) {
3097+
const source = fs.readFileSync(file, "utf8");
3098+
const specifiers = new Set<string>();
3099+
for (const pattern of VALUE_IMPORT_PATTERNS) {
3100+
for (const match of source.matchAll(pattern)) {
3101+
if (match[1]) {
3102+
specifiers.add(match[1]);
3103+
}
3104+
}
3105+
}
3106+
for (const specifier of specifiers) {
3107+
const packageName = packageNameFromBareSpecifier(specifier);
3108+
if (!packageName) {
3109+
continue;
3110+
}
3111+
if (nodeBuiltins.has(packageName)) {
3112+
continue;
3113+
}
3114+
if (packageName === "openclaw" || packageName.startsWith("@openclaw/")) {
3115+
continue;
3116+
}
3117+
if (mirroredCore.has(packageName) || extensionDeps.has(packageName)) {
3118+
continue;
3119+
}
3120+
if (KNOWN_UNMIRRORED_BARE_IMPORTS.has(packageName)) {
3121+
continue;
3122+
}
3123+
if (!rootDeps.has(packageName)) {
3124+
// Not a root runtime dep; not our concern (could be a peer/dev import
3125+
// that resolves through some other path; the mirror does not own it).
3126+
continue;
3127+
}
3128+
if (!violations.has(packageName)) {
3129+
violations.set(packageName, path.relative(repoRoot, file).replaceAll(path.sep, "/"));
3130+
}
3131+
}
3132+
}
3133+
3134+
if (violations.size > 0) {
3135+
const summary = [...violations.entries()]
3136+
.toSorted(([left], [right]) => left.localeCompare(right))
3137+
.map(([packageName, filePath]) => ` - ${packageName} (e.g. ${filePath})`)
3138+
.join("\n");
3139+
throw new Error(
3140+
[
3141+
"Bare imports found in src/ that are root-package runtime deps but are neither",
3142+
"in MIRRORED_CORE_RUNTIME_DEP_NAMES nor declared by any extension's package.json.",
3143+
"These will be missing from the runtime-deps mirror at gateway start and Node",
3144+
"will fail to resolve them. Either add the package to MIRRORED_CORE_RUNTIME_DEP_NAMES,",
3145+
"declare it under an owning extension's dependencies, or add it to",
3146+
"KNOWN_UNMIRRORED_BARE_IMPORTS in this test with a comment explaining why.",
3147+
"",
3148+
summary,
3149+
].join("\n"),
3150+
);
3151+
}
3152+
});
3153+
});

‎src/plugins/bundled-runtime-deps.ts‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,20 @@ const DEFAULT_UNKNOWN_RUNTIME_DEPS_MIN_AGE_MS = 10 * 60_000;
7272
const BUNDLED_RUNTIME_DEPS_INSTALL_PROGRESS_INTERVAL_MS = 5_000;
7373
const BUNDLED_RUNTIME_MIRROR_MATERIALIZED_EXTENSIONS = new Set([".cjs", ".js", ".mjs"]);
7474
const BUNDLED_EXTENSION_DIST_DIR = "extensions";
75-
const MIRRORED_CORE_RUNTIME_DEP_NAMES = ["semver", "tslog"] as const;
75+
const MIRRORED_CORE_RUNTIME_DEP_NAMES = [
76+
"@agentclientprotocol/sdk",
77+
"@lydell/node-pty",
78+
"croner",
79+
"dotenv",
80+
"jiti",
81+
"json5",
82+
"jszip",
83+
"markdown-it",
84+
"semver",
85+
"tar",
86+
"tslog",
87+
"web-push",
88+
] as const;
7689
const MIRRORED_PACKAGE_RUNTIME_DEP_PLUGIN_ID = "openclaw-core";
7790
const BUNDLED_RUNTIME_MIRROR_PLUGIN_REGION_RE = /(?:^|\n)\/\/#region extensions\/[^/\s]+(?:\/|$)/u;
7891
const BUNDLED_RUNTIME_MIRROR_IMPORT_SPECIFIER_RE =

0 commit comments

Comments
 (0)