Skip to content

fix(plugins): reject hollow installs and repair missing dependencies#103398

Open
nxmxbbd wants to merge 15 commits into
openclaw:mainfrom
nxmxbbd:fix/101049-plugin-install-dep-gate
Open

fix(plugins): reject hollow installs and repair missing dependencies#103398
nxmxbbd wants to merge 15 commits into
openclaw:mainfrom
nxmxbbd:fix/101049-plugin-install-dep-gate

Conversation

@nxmxbbd

@nxmxbbd nxmxbbd commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Closes #101049

What Problem This Solves

Fixes an issue where installing a channel plugin through the managed npm flow could report success with an empty or incomplete dependency tree. The plugin then appeared installed and loaded at startup, but its provider failed when importing a missing runtime dependency. In the reported Microsoft Teams case, webhook deliveries failed while the local install still looked healthy, and only a manual npm reinstall inside the managed project restored the channel.

Why This Change Was Made

The managed npm install path verified lockfile metadata, platform packages, and peer links after npm install, but it did not verify that the installed plugin's own declared runtime dependencies resolved from the installed package root. Doctor likewise did not classify an installed-but-hollow dependency tree as repairable, even though plugin status already exposed the dependency state.

This change reuses the existing plugin dependency-status machinery in both paths:

  • Managed npm installs now reject a package when a required declared dependency is absent after npm exits successfully. The error names the missing packages and uses the existing managed-root rollback path.
  • A dependency counts as materialized only when its package root contains the package.json npm places in published packages, so an empty residual directory is not mistaken for an installed dependency.
  • Doctor checks active source:npm plugin roots that canonically match their persisted install record, names missing packages in core/doctor/configured-plugin-installs, and sends affected installs through the retention-marker-backed package updater. This includes catalog-backed official installs, so same-version repairs activate a fresh generation instead of reusing the hollow tree.
  • Dependency normalization follows npm semantics where optionalDependencies override same-name dependencies, so packages that intentionally make a dependency optional are not rejected when npm legitimately omits it.

Missing optional dependencies do not fail either path. Resolution follows Node walk-up semantics, so a required dependency hoisted to the managed project root remains valid. Import probing at every gateway startup is outside this change; the install path covers new writes and doctor covers persisted corruption.

User Impact

A managed plugin install with a hollow dependency tree now fails loudly instead of leaving a plugin that appears healthy but cannot run. Existing corrupted npm installs are reported by doctor with the missing package names, and doctor --fix reinstalls the package into a healthy generation instead of requiring manual npm commands in an internal managed directory. Packages that list the same package in both dependencies and optionalDependencies keep npm's override behavior and remain installable when that dependency is absent.

Evidence

Full E2E: partial. The persisted-corruption detection and repair path is direct CLI E2E. The install-time hollow-tree fault remains a production-function integration test because real npm cannot be made to deterministically exit successfully while omitting a required dependency.

Atomic proof status

  • Reject hollow install --- production-function integration: the regression calls the production npm installer; only the npm subprocess/materialization result is fault-injected. It is RED on the base behavior (ok: true) and GREEN here with the missing package named.
  • Rollback failed managed root --- production-function integration + direct filesystem after-state: the same regression verifies that the failed plugin package root no longer exists after rejection.
  • Preserve npm optional override semantics --- production-function integration + real npm contract check: a regression installs a package declaring the same absent package as both required and optional, and the production installer accepts it. A direct npm check also confirmed that optionalDependencies can override same-name dependencies and that npm may exit successfully while omitting the package.
  • Detect a corrupted installed tree --- direct E2E: a real CLI install was corrupted by removing its plugin-local ms, after which focused doctor returned one finding naming ms.
  • Repair the corrupted installed tree --- direct E2E: from an already-corrupt managed generation, real doctor --fix fetched npm metadata and the package tarball, switched the install record to a fresh activation generation, restored [email protected], and left focused doctor clean.
  • Retain official same-version repairs --- production-function regression: the official catalog path was RED because it bypassed the retained updater; it now routes through the marker-backed updater and verifies a fresh repair generation.
  • Preserve detection across official-package mismatch --- production-function regression: an active npm record whose configured id also mapped to a different official package was RED with no configured-install finding; detection now stays on the record-backed route and names the missing dependency.
  • Fail closed when retention cannot be prepared --- production-function regression: a real marker-directory filesystem failure was RED because doctor still invoked the updater; the affected dependency repair is now warned and skipped before any package update.
  • Report skipped dependency repairs --- production-function regression: a valid npm record without spec was RED because doctor cleared its marker but emitted no warning or failed id; mixed-batch coverage now reports that failure while an unrelated repair completes and keeps its retained marker.
  • Reject empty dependency directories --- production-function regression: a real Node resolution probe returns MODULE_NOT_FOUND for the residual directory; install and doctor/status regressions now require the dependency package manifest.
  • Healthy npm installs --- direct real-npm non-regression: real managed npm installs with complete nested dependency trees passed the new gate.

Real behavior proof

  • Behavior or issue addressed: installed npm plugins whose required dependency tree is missing or later becomes corrupted, while preserving npm-valid optional dependency overrides.
  • Real environment tested: Linux source checkout, Node v22.22.1, isolated OPENCLAW_STATE_DIR, and a localhost-only npm registry serving real metadata and a tarball for [email protected] with bundled [email protected].
  • Exact steps or command run after this patch: start from a genuine source:npm install record whose active path is already a managed generation; move its plugin-local node_modules/ms aside; run focused doctor lint; run doctor --fix --yes --non-interactive --no-workspace-suggestions with npm pointed at the fresh local registry; independently read the updated SQLite record and resolve the restored dependency; rerun focused doctor lint.
  • Evidence after fix: copied terminal/registry/database output from the isolated run, with the state path shortened to $STATE_DIR:
$ openclaw doctor --lint --json --only core/doctor/configured-plugin-installs
{"ok":false,"checksRun":1,"checksSkipped":50,"findings":[{"checkId":"core/doctor/configured-plugin-installs","severity":"warning","message":"Configured plugin dep-proof-demo is installed but required dependencies are missing from its dependency tree: ms.","target":"dep-proof-demo",...}]}

$ openclaw doctor --fix --yes --non-interactive --no-workspace-suggestions
Repaired broken installed plugin "dep-proof-demo".

# Fresh-cache registry requests during doctor repair
GET /dep-proof-demo                                      200
GET /dep-proof-demo/-/dep-proof-demo-1.0.0.tgz          200

# Independent read of installed_plugin_index after repair
source: npm
spec: [email protected]
integrity/shasum: present
installPath: $STATE_DIR/npm/projects/dep-proof-demo__openclaw-generation__g-2e66e24c4f3583e3/node_modules/dep-proof-demo

# Independent package read from the active generation
ms 2.1.3

$ openclaw doctor --lint --json --only core/doctor/configured-plugin-installs
{"ok":true,"checksRun":1,"checksSkipped":50,"findings":[]}
  • Observed result after fix: doctor identified the deliberately removed required package, the repair used the real npm updater and generation swap, the active tree resolved the exact restored package version, and the same focused check became clean.
  • What was not tested: a live Microsoft Teams webhook round-trip, because the changed boundary is plugin installation/repair rather than Bot Framework delivery. A naturally hollow successful npm run was not reproduced; install rejection therefore remains mock-fault integration proof and the aggregate proof classification remains partial rather than full E2E.

Current main merge refresh

The branch carries two signed merge commits after the reviewed head. Merge 704e5a2fbfd35ceefb79664f88a4d77c783b07ba refreshed the then-current main 4ae4b5c2a464608045e29fe72793dbb6fbbcdad7 with the reviewed six-file patch byte-identical. Merge d5ebd2c9faf5e19d4b7a48ec102e46c59a1ae211 resolves conflicts against current main 9497450511d2996dae6632305ee2111a93377657, where the plugin install split (#107960) moved the managed npm install flow out of src/plugins/install.ts into focused modules. The install-time dependency gate now sits at the same anchor inside installPluginFromManagedNpmRoot in src/plugins/install-managed-npm.ts; the ported import and gate lines are byte-identical to the reviewed change, src/plugins/install.ts is byte-identical to main's re-export barrel and leaves the diff, and the other five changed files carry the reviewed change merged with current main.

The exact merged head passed the direct affected suites and repository changed-file gate:

src/plugins/install.npm-spec.test.ts                                  65 passed
src/commands/doctor/shared/missing-configured-plugin-install.test.ts  95 passed
src/plugins/status.registry-snapshot.test.ts                            5 passed
check:changed --base 9497450511d2996dae6632305ee2111a93377657        core + coreTests passed
oxfmt changed files                                                    clean
oxlint changed files                                                   clean

check:changed included core production/test typechecks, conflict-marker and dependency guards, changed-file lint, database-first/media/runtime-sidecar guards, and runtime import-cycle checks. One lane is excluded: the max-lines ratchet fails via a stale main baseline entry for src/agents/modes/interactive/theme/theme.ts (file and baseline byte-identical to main here); all other lanes passed with that entry pruned locally. Hosted checks for this new head are pending and should be judged from this head's check runs.

Scope, compatibility, and risk

No dependency, lockfile, configuration schema, authentication, permissions, network access, or persisted-record schema changes are introduced. The behavior change is limited to rejecting required dependencies that do not resolve after a managed npm install, preserving npm's optional override semantics during dependency status normalization, and treating the same persisted missing-required condition as a repairable doctor finding for the active matching npm install record. Path/archive records and same-id configured shadows are not advertised as auto-repairable. Optional dependencies, same-name optional overrides, and valid hoisted dependencies have explicit non-regression coverage. The main risks are false attribution and reusing an already-hollow same-version project; canonical root/source eligibility tests, fresh-generation retention tests, real nested-dependency installs, optional-override coverage, and direct repeated-generation repair E2E cover those boundaries.

AI-assisted; contributor manually reviewed and is responsible for the change.

nxmxbbd added 11 commits July 8, 2026 02:46
…dencies

npm can exit 0 while leaving the installed plugin's declared dependency
tree unmaterialized; the hollow install previously passed lock-metadata
verification and loaded at startup only to die at import time. Verify the
declared required dependencies resolve from the installed package dir and
roll the managed root back when they do not.
…d dependencies

Status surfaces already point users at doctor --fix when an installed
plugin's dependency tree is broken, but doctor only recognized entry
diagnostics and stale runtime packages as repairable. Walk record-backed
snapshot plugins with buildPluginDependencyStatus, surface the missing
package names in the health finding, and route the plugin through the
existing repairable-install flows so --fix reinstalls the package.
@openclaw-barnacle openclaw-barnacle Bot added commands Command implementations size: L labels Jul 10, 2026
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P0 Emergency: data loss, security bypass, crash loop, or unusable core runtime. labels Jul 10, 2026
@clawsweeper

clawsweeper Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed July 15, 2026, 11:50 PM ET / July 16, 2026, 03:50 UTC.

Summary
The PR rejects managed npm plugin installs with unresolved required dependencies, treats empty dependency directories as missing, and extends doctor to detect and reinstall matching corrupted active npm installs into a fresh generation.

PR surface: Source +202, Tests +741. Total +943 across 6 files.

Reproducibility: yes. from source and supplied evidence, though not as a newly executed current-main failure in this review. An active managed npm record can point to a plugin tree missing a declared required dependency, and the contributor demonstrates that persisted corruption through the real doctor path.

Review metrics: none identified.

Stored data model
Persistent data-model change detected: migration/backfill/repair: src/commands/doctor/shared/missing-configured-plugin-install.test.ts, migration/backfill/repair: src/commands/doctor/shared/missing-configured-plugin-install.ts, serialized state: src/commands/doctor/shared/missing-configured-plugin-install.test.ts, serialized state: src/plugins/status.registry-snapshot.test.ts, unknown-data-model-change: src/commands/doctor/shared/missing-configured-plugin-install.test.ts, unknown-data-model-change: src/commands/doctor/shared/missing-configured-plugin-install.ts. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #101049
Summary: This PR is the direct candidate fix for the canonical hollow managed npm installation issue; the installer split is adjacent refactoring rather than a competing fix.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Risk before merge

  • [P1] The exact PR head is behind current main, so the final merge result should be refreshed and reviewed against the recently reorganized plugin installer modules.
  • [P1] A required test shard currently fails, and the supplied context does not identify its failing test; maintainers should resolve or attribute that failure before merge.

Maintainer options:

  1. Decide the mitigation before merge
    Land the install-time dependency gate and record-scoped doctor repair after refreshing onto current main and obtaining a green or clearly unrelated exact-head required-check result.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • [P2] Refresh the branch onto current main, inspect the failing required shard, and proceed with normal exact-head landing review if the failure is resolved or proven unrelated.

Security
Cleared: The patch introduces no new dependency source, lockfile, lifecycle script, permission, secret, network authority, or persisted schema and reuses existing managed updater paths.

Review details

Best possible solution:

Land the install-time dependency gate and record-scoped doctor repair after refreshing onto current main and obtaining a green or clearly unrelated exact-head required-check result.

Do we have a high-confidence way to reproduce the issue?

Yes from source and supplied evidence, though not as a newly executed current-main failure in this review. An active managed npm record can point to a plugin tree missing a declared required dependency, and the contributor demonstrates that persisted corruption through the real doctor path.

Is this the best way to solve the issue?

Yes. Checking dependencies while the managed install rollback boundary remains active and repairing existing record-backed corruption through doctor is narrower and safer than gateway-startup import probing or a parallel installer.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 957cc81175a3.

Label changes

Label justifications:

  • P0: The underlying defect can block a community user's channel setup and first useful delivery while appearing healthy, with recovery requiring manual npm commands inside an internal managed directory.
  • rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): After-fix live output demonstrates the changed doctor behavior on a real isolated npm install record, including detection, actual registry fetches, fresh-generation activation, restored package resolution, and a clean repeat check.
  • proof: sufficient: Contributor real behavior proof is sufficient. After-fix live output demonstrates the changed doctor behavior on a real isolated npm install record, including detection, actual registry fetches, fresh-generation activation, restored package resolution, and a clean repeat check.
Evidence reviewed

PR surface:

Source +202, Tests +741. Total +943 across 6 files.

View PR surface stats
Area Files Added Removed Net
Source 3 259 57 +202
Tests 3 744 3 +741
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 6 1003 60 +943

What I checked:

  • Canonical bug and user impact: The linked report describes a managed Microsoft Teams npm install with missing runtime dependencies that appeared installed but failed webhook delivery until the user manually reinstalled dependencies inside the managed project.
  • Install-time enforcement: The PR head reuses the existing dependency-status machinery after managed npm installation and invokes the existing rollback path when a declared required dependency does not resolve from the installed plugin root. (src/plugins/install-managed-npm.ts:537, d5ebd2c9faf5)
  • npm dependency semantics: The normalizer honors npm's same-name optional dependency override, while dependency materialization now requires a package manifest rather than accepting an empty residual directory. (src/plugins/status-dependencies-core.ts:49, d5ebd2c9faf5)
  • Doctor repair boundary: Doctor limits this repair classification to an active plugin root that canonically matches a persisted source:npm install record and routes the repair through retained managed-generation update behavior. (src/commands/doctor/shared/missing-configured-plugin-install.ts:668, d5ebd2c9faf5)
  • Regression coverage: The changed tests cover missing and empty dependencies, optional overrides, hoisted dependencies, active-record matching, official-package mismatch, retention preparation failures, partial batch failures, and fresh-generation repair. (src/commands/doctor/shared/missing-configured-plugin-install.test.ts:3571, d5ebd2c9faf5)
  • Real behavior proof: The PR body provides an after-fix isolated npm run where doctor identifies the removed ms package, performs real registry fetches, activates a fresh generation, independently resolves [email protected], and then reports a clean repeat check. (d5ebd2c9faf5)

Likely related people:

  • steipete: The merged plugin installation split moved the managed npm flow into the module changed by this PR while preserving its lifecycle and rollback behavior. (role: recent installer refactor contributor; confidence: high; commits: 3eb8b3a0f838; files: src/plugins/install-managed-npm.ts, src/plugins/install.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (2 earlier review cycles)
  • reviewed 2026-07-10T10:44:39.903Z sha 7d283eb :: needs maintainer review before merge. :: none
  • reviewed 2026-07-10T14:35:21.420Z sha 704e5a2 :: needs maintainer review before merge. :: none

Merge current upstream main into PR openclaw#103398 while retaining the
validated install and doctor corrections from the refresh.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

commands Command implementations P0 Emergency: data loss, security bypass, crash loop, or unusable core runtime. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. size: XL status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: msteams plugin npm install can leave an empty dependency tree — plugin loads, provider dies silently, Microsoft sees delivery errors

1 participant