Skip to content

fix(plugins): stage bundled-plugin runtime-dep install outside the plugin root#70852

Merged
steipete merged 5 commits into
openclaw:mainfrom
simonemacario:fix/bundled-plugin-docker-workspace-protocol
Apr 24, 2026
Merged

fix(plugins): stage bundled-plugin runtime-dep install outside the plugin root#70852
steipete merged 5 commits into
openclaw:mainfrom
simonemacario:fix/bundled-plugin-docker-workspace-protocol

Conversation

@simonemacario

Copy link
Copy Markdown
Contributor

Summary

Fixes gateway-startup plugin-runtime-dep install failure on packaged (Docker image, npm global) installs of 2026.4.22 where every bundled plugin whose package.json still contains a workspace:* dep fails with:

npm error code EUNSUPPORTEDPROTOCOL
npm error Unsupported URL Type "workspace:": workspace:*

Reported in #70844 (Docker image, EUNSUPPORTEDPROTOCOL at install step); same root cause as #70701, #70756, #70773, #70818, #70839 (npm global, downstream Cannot find package 'openclaw' from plugin-runtime-deps/… at ESM import).

Root cause

ensureBundledPluginRuntimeDeps in src/plugins/bundled-runtime-deps.ts already filters workspace: specs from the CLI argument list (normalizeInstallableRuntimeDepVersion, parseInstallableRuntimeDep). But the install step is spawnSync("npm", ["install", ...specs], { cwd: installExecutionRoot }), and installExecutionRoot defaults to installRoot. When installRoot === pluginRoot (the packaged case where the plugin dir is writable and no external stage is configured), npm reads the plugin's own package.json as the project manifest and tries to resolve its workspace:* deps — failing the whole batch regardless of what was passed on the CLI.

Evidence: upstream/main carries workspace:* declarations in 106 extensions/*/package.json (predominantly "@openclaw/plugin-sdk": "workspace:*"). The 18 plugins that fail in the Docker repro are exactly the subset that declare such entries as runtime deps (dependencies / optionalDependencies).

The npm-global variants in #70701 / #70756 / #70773 / #70818 / #70839 hit a related but different surface: resolveBundledRuntimeDependencyInstallRoot resolves to an external stage dir (clean manifest), so npm install succeeds but the resulting node_modules/ tree does not satisfy the filtered-out workspace packages, and the subsequent ESM import of openclaw fails at plugin load. Same defect class, different failure point.

Fix

Activate the existing isolated-execution-root + replaceNodeModulesDir machinery for the packaged case too. When installRoot === pluginRoot and we are not in the source-checkout cache-dir path, stage the install inside <installRoot>/.openclaw-install-stage (minimal generated package.json) and move the produced node_modules/ back to the plugin root as before.

+const isPluginRootInstall =
+  path.resolve(installRoot) === path.resolve(params.pluginRoot);
+const sourceCheckoutCacheStage =
+  cacheDir &&
+  isPluginRootInstall &&
+  resolveSourceCheckoutBundledPluginPackageRoot(params.pluginRoot)
+    ? cacheDir
+    : undefined;
 const installExecutionRoot =
-  cacheDir &&
-  path.resolve(installRoot) === path.resolve(params.pluginRoot) &&
-  resolveSourceCheckoutBundledPluginPackageRoot(params.pluginRoot)
-    ? cacheDir
-    : undefined;
+  sourceCheckoutCacheStage ??
+  (isPluginRootInstall ? path.join(installRoot, PLUGIN_ROOT_INSTALL_STAGE_DIR) : undefined);

No change to the existing workspace: filter, no new surface, no change to external stage or source-checkout semantics.

Shape note

@steipete flagged in #70138 that runtime-dep repair belongs at gateway/plugin startup (not npm pack postinstall). This change keeps that contract — it only widens when the existing stage/move is activated; the repair still runs through ensureBundledPluginRuntimeDeps called from src/plugins/loader.ts:2190. It does not reintroduce the eager-install shape that was closed in #70650.

Validation

  • pnpm exec vitest run --config test/vitest/vitest.plugins.config.ts src/plugins/bundled-runtime-deps.test.ts — 31/31 (includes 1 new regression test: stages plugin-root install when the plugin's own package.json declares workspace:* deps)
  • pnpm exec vitest run --config test/vitest/vitest.plugins.config.ts src/plugins/ — 43/43 across 8 test files
  • pnpm exec tsgo --noEmit -p tsconfig.core.json — clean
  • pnpm exec tsgo --noEmit -p tsconfig.core.test.json — clean
  • pnpm exec oxlint src/plugins/bundled-runtime-deps.ts src/plugins/bundled-runtime-deps.test.ts — 0 warnings, 0 errors
  • node scripts/check-src-extension-import-boundary.mjs --json[]
  • node scripts/check-sdk-package-extension-import-boundary.mjs --json[]

I did not run codex review --base origin/main because I do not have Codex CLI access locally; happy to run Codex review in a later iteration if a maintainer provides guidance.

Test plan

AI assistance

This PR was AI-assisted with Claude Code (Opus 4.7). All code paths changed here were read end-to-end by me; the diagnosis against src/plugins/loader.ts:2190 and src/plugins/bundled-runtime-deps.ts is documented in the linked issue #70844 comment thread.

Testing degree: fully tested for the touched bundled-plugin runtime-dep install-staging surface.

I understand the code path changed here: packaged bundled plugins now stage their runtime-dep install one directory below pluginRoot so npm never reads the plugin's workspace:*-containing manifest during install; the existing replaceNodeModulesDir helper moves the produced node_modules/ back to pluginRoot after install succeeds.

Closes #70844

…ugin root

When a packaged bundled plugin's `pluginRoot` is used directly as the npm
execution cwd, `npm install <specs>` resolves the plugin's own
`package.json` as the project manifest and fails with
`EUNSUPPORTEDPROTOCOL: Unsupported URL Type "workspace:": workspace:*`
whenever that manifest declares a `workspace:` runtime dep (e.g.
`"@openclaw/plugin-sdk": "workspace:*"`). This takes out every plugin
with any runtime deps at gateway startup.

`ensureBundledPluginRuntimeDeps` already filters `workspace:` specs from
the CLI arguments, but npm's own resolver reads the cwd manifest
regardless, so the filter alone is not enough. The existing isolated
execution-root + `replaceNodeModulesDir` machinery handles this exact
problem for source-checkout + cache-hit installs. This change activates
the same staging path for the packaged case: when `installRoot ===
pluginRoot` and we are not in the source-checkout cache path, stage the
install inside `<pluginRoot>/.openclaw-install-stage` (which has a
minimal generated `package.json`) and move the produced `node_modules/`
back to the plugin root as before.

- Add regression test `stages plugin-root install when the plugin's own
  package.json declares workspace:* deps` covering the Docker scenario
  (mixed `workspace:*` + concrete runtime dep, e.g. anthropic-style
  `@openclaw/plugin-sdk` + `@anthropic-ai/sdk`).
- Update existing plugin-root-install expectations (`installs
  plugin-local runtime deps when one is missing`, `skips workspace-only
  runtime deps before npm install`, `installs deps that are only present
  in the package root`, `does not trust runtime deps that only resolve
  from the package root`, `does not treat sibling extension runtime deps
  as satisfying a plugin`) to assert the new `installExecutionRoot`.

Reported in openclaw#70844; same root cause as openclaw#70701, openclaw#70756, openclaw#70773, openclaw#70818,
openclaw#70839 which see the downstream "Cannot find package 'openclaw' from
plugin-runtime-deps" symptom because their
`resolveBundledRuntimeDependencyInstallRoot` resolves to an external
stage dir (clean manifest) so the install succeeds but the resulting
node_modules tree cannot satisfy the filtered-out workspace packages at
ESM import time.

## AI assistance

This PR was AI-assisted with Claude Code.

Testing degree: fully tested for the touched `bundled-runtime-deps`
install staging surface.

- `pnpm exec vitest run --config test/vitest/vitest.plugins.config.ts src/plugins/bundled-runtime-deps.test.ts` (31/31)
- `pnpm exec vitest run --config test/vitest/vitest.plugins.config.ts src/plugins/` (43/43 across 8 files)
- `pnpm exec tsgo --noEmit -p tsconfig.core.json`, `pnpm exec tsgo --noEmit -p tsconfig.core.test.json` (clean)
- `pnpm exec oxlint src/plugins/bundled-runtime-deps.ts src/plugins/bundled-runtime-deps.test.ts` (0 warnings, 0 errors)
- `node scripts/check-src-extension-import-boundary.mjs --json` and `node scripts/check-sdk-package-extension-import-boundary.mjs --json` (both `[]`)

I understand the code path changed here: packaged bundled plugins now
stage their runtime-dep install one directory below `pluginRoot` so npm
never reads the plugin's `workspace:*`-containing manifest during
install; after install completes, the produced `node_modules/` is moved
back to `pluginRoot` via the existing `replaceNodeModulesDir` helper.

Signed-off-by: Simone Macario <[email protected]>
@greptile-apps

greptile-apps Bot commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes the EUNSUPPORTEDPROTOCOL crash at gateway startup for packaged (Docker/npm-global) installs by routing npm install through an isolated .openclaw-install-stage/ subdirectory instead of the plugin root, preventing npm from picking up workspace:* entries in the plugin's own package.json. The logic is correct and the regression test is well-targeted.

  • Disk leak in stage directory: installBundledRuntimeDeps never removes installExecutionRoot after replaceNodeModulesDir copies node_modules out. The full npm-produced tree (potentially hundreds of MB across 18+ affected plugins) stays under .openclaw-install-stage/ indefinitely. A finally-guarded fs.rmSync(installExecutionRoot, { recursive: true, force: true }) after the move would close this gap.

Confidence Score: 3/5

Fix is functionally correct but leaves a persistent duplicate node_modules tree in the stage directory for every plugin that undergoes a runtime-dep install.

The core isolation mechanism works and passes all 31 tests. However, the missing cleanup of .openclaw-install-stage/ is a real resource-management defect introduced on the new code path — it silently doubles disk usage per affected bundled plugin on every fresh install, which is non-trivial for Docker images and npm-global deployments.

src/plugins/bundled-runtime-deps.ts — specifically the post-move cleanup path in installBundledRuntimeDeps (lines 862–868)

Comments Outside Diff (1)

  1. src/plugins/bundled-runtime-deps.ts, line 862-868 (link)

    P1 Stage directory not cleaned up after install

    After replaceNodeModulesDir copies node_modules out of .openclaw-install-stage/, the stage directory itself (including its full npm-produced node_modules/ tree and the generated package.json) is never removed. replaceNodeModulesDir uses fs.cpSync (not a rename), so the source tree stays on disk indefinitely.

    For a Docker or npm-global install with multiple bundled plugins, each plugin that requires a runtime-dep install will leave a duplicate node_modules/ under .openclaw-install-stage/ — potentially tens to hundreds of MB per plugin — that survives every subsequent gateway restart. The PR test plan notes "verify cleanup", but there is no corresponding fs.rmSync(installExecutionRoot, …) in the success or error path of installBundledRuntimeDeps.

    Consider adding a finally block (or a best-effort cleanup after replaceNodeModulesDir) to remove installExecutionRoot when it differs from installRoot.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: src/plugins/bundled-runtime-deps.ts
    Line: 862-868
    
    Comment:
    **Stage directory not cleaned up after install**
    
    After `replaceNodeModulesDir` copies `node_modules` out of `.openclaw-install-stage/`, the stage directory itself (including its full npm-produced `node_modules/` tree and the generated `package.json`) is never removed. `replaceNodeModulesDir` uses `fs.cpSync` (not a rename), so the source tree stays on disk indefinitely.
    
    For a Docker or npm-global install with multiple bundled plugins, each plugin that requires a runtime-dep install will leave a duplicate `node_modules/` under `.openclaw-install-stage/` — potentially tens to hundreds of MB per plugin — that survives every subsequent gateway restart. The PR test plan notes "verify cleanup", but there is no corresponding `fs.rmSync(installExecutionRoot, …)` in the success or error path of `installBundledRuntimeDeps`.
    
    Consider adding a `finally` block (or a best-effort cleanup after `replaceNodeModulesDir`) to remove `installExecutionRoot` when it differs from `installRoot`.
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/plugins/bundled-runtime-deps.ts
Line: 862-868

Comment:
**Stage directory not cleaned up after install**

After `replaceNodeModulesDir` copies `node_modules` out of `.openclaw-install-stage/`, the stage directory itself (including its full npm-produced `node_modules/` tree and the generated `package.json`) is never removed. `replaceNodeModulesDir` uses `fs.cpSync` (not a rename), so the source tree stays on disk indefinitely.

For a Docker or npm-global install with multiple bundled plugins, each plugin that requires a runtime-dep install will leave a duplicate `node_modules/` under `.openclaw-install-stage/` — potentially tens to hundreds of MB per plugin — that survives every subsequent gateway restart. The PR test plan notes "verify cleanup", but there is no corresponding `fs.rmSync(installExecutionRoot, …)` in the success or error path of `installBundledRuntimeDeps`.

Consider adding a `finally` block (or a best-effort cleanup after `replaceNodeModulesDir`) to remove `installExecutionRoot` when it differs from `installRoot`.

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "fix(plugins): stage bundled-plugin runti..." | Re-trigger Greptile

@steipete

Copy link
Copy Markdown
Contributor

Codex review: this is the right canonical fix for the Discord/Telegram/ACP packaged runtime-dep duplicates I just de-duped into #70844.

One merge blocker remains: CI check-lint is red from oxlint on the new test. The failing assertion is almost certainly this line:

expect(path.resolve(calls[0]!.installExecutionRoot!)).not.toEqual(path.resolve(pluginRoot));

calls is typed as BundledRuntimeDepsInstallParams[], and installExecutionRoot is already present after the equality assertion above, so oxlint treats the non-null assertion as unnecessary. Rewriting it through an explicit local value should clear lint, e.g. const installExecutionRoot = calls[0]?.installExecutionRoot; expect(installExecutionRoot).toBeDefined(); expect(path.resolve(installExecutionRoot!)).not..., or compare the expected call object directly and drop the extra path assertion if you prefer.

The important part to keep in the PR: packaged plugin-root installs must run from .openclaw-install-stage, while source-checkout cache installs still use the cache stage.

@openclaw-barnacle openclaw-barnacle Bot added scripts Repository scripts docker Docker and sandbox tooling size: M and removed size: S labels Apr 24, 2026
@steipete

Copy link
Copy Markdown
Contributor

Codex follow-up pushed on top of the PR.

What changed:

  • fixed the lint blocker from the unnecessary non-null assertion in src/plugins/bundled-runtime-deps.test.ts
  • fixed the staged install cleanup gap so .openclaw-install-stage does not keep a duplicate node_modules
  • fixed one remaining npm-global/root-owned path: external staged installs must create the openclaw/plugin-sdk alias in the mirrored dist root, not in the unwritable packaged dist/extensions/node_modules
  • fixed the Docker root-owned harness so gateway log redirection runs as appuser

Validation I ran locally:

  • pnpm check:changed
  • pnpm test src/plugins/bundled-runtime-deps.test.ts
  • pnpm test src/plugins/loader.test.ts -- -t "loads bundled plugins with plugin-sdk imports from an external stage dir"
  • pnpm test:docker:bundled-channel-deps through fresh Telegram/Discord/Slack/Feishu/memory-lancedb and update scenarios; the first run exposed the root-owned external-stage miss above
  • root-owned Docker slice rerun after the fix: passed, Slack visible from external staged deps
  • setup-entry + load-failure Docker slices rerun after the fix: passed

Remote CI is queued/running on the latest SHA now.

@steipete
steipete merged commit dd15762 into openclaw:main Apr 24, 2026
84 checks passed
@steipete

Copy link
Copy Markdown
Contributor

Landed via rebase onto main.

  • Gate: pnpm check:changed; bundled runtime Docker fresh/update/root-owned/setup-entry/load-failure slices; CI product/code gates green at merge.
  • Source head: simonemacario@fdac137
  • Landed commit: dd15762

Thanks @simonemacario!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docker Docker and sandbox tooling scripts Repository scripts size: M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: v2026.4.22 official Docker image — 18 bundled plugins fail runtime-dep install with EUNSUPPORTEDPROTOCOL: "workspace:*"

2 participants