Skip to content

fix(memory-lancedb): align apache arrow peer dependency#90407

Closed
sahibzada-allahyar wants to merge 1 commit into
openclaw:mainfrom
sahibzada-allahyar:tui-90295-lancedb-peer-fix
Closed

fix(memory-lancedb): align apache arrow peer dependency#90407
sahibzada-allahyar wants to merge 1 commit into
openclaw:mainfrom
sahibzada-allahyar:tui-90295-lancedb-peer-fix

Conversation

@sahibzada-allahyar

@sahibzada-allahyar sahibzada-allahyar commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Pin extensions/memory-lancedb from [email protected] to 18.1.0, the highest version accepted by @lancedb/[email protected].
  • Regenerate the plugin npm-shrinkwrap.json with the repo shrinkwrap generator (scripts/generate-npm-shrinkwrap.mjs) so published npm installs use consistent Arrow 18 transitive metadata.
  • Update the root pnpm-lock.yaml to match the package manifest.

Fixes #90295.

Real Behavior Proof

Behavior or issue addressed: @openclaw/memory-lancedb cannot install under npm 11. Its manifest pinned [email protected], but @lancedb/[email protected] only accepts apache-arrow >=15.0.0 <=18.1.0, so npm 11 peer resolution aborts with ERESOLVE. This PR pins apache-arrow to 18.1.0 (the top of the accepted range) and regenerates the plugin shrinkwrap and root lockfile to match.

Real environment tested: Reproduced locally on macOS (arm64) against the live npm registry with npm 11.12.1 / Node v26 / pnpm 11.2.2 — the npm 11 / pnpm 11.2.2 line this repo targets. The two installs below use the real published @lancedb/[email protected] package with apache-arrow at each version, isolating the exact peer constraint the manifest change resolves.

Exact steps or command run after this patch:

npm view @lancedb/[email protected] peerDependencies --json
# before (current main manifest): apache-arrow 21.1.0
npm install --package-lock-only --ignore-scripts   # in a project requiring lancedb 0.30.0 + [email protected]
# after (this PR's manifest): apache-arrow 18.1.0
npm install --package-lock-only --ignore-scripts   # in a project requiring lancedb 0.30.0 + [email protected]

Evidence after fix: Copied terminal output from the runs above.

Live peer contract from the registry (the version ceiling the fix targets):

$ npm view @lancedb/[email protected] peerDependencies --json
{
  "apache-arrow": ">=15.0.0 <=18.1.0"
}

Before — [email protected] (what main declares) is rejected by npm 11:

npm error code ERESOLVE
npm error While resolving: [email protected]
npm error Found: [email protected]
npm error Could not resolve dependency:
npm error peer apache-arrow@">=15.0.0 <=18.1.0" from @lancedb/[email protected]
# npm exited 1

After — [email protected] (what this PR declares) resolves cleanly:

up to date, audited 31 packages in 2s
found 0 vulnerabilities
# npm exited 0

Observed result after fix: With [email protected] npm 11 aborts (ERESOLVE, exit 1); with [email protected] it resolves the tree and exits 0. This matches the manifest change in this PR (extensions/memory-lancedb/package.json: apache-arrow 21.1.018.1.0), so the regenerated graph installs under npm 11 where main does not.

What was not tested: Runtime LanceDB query behavior on a physical device, and the full 152-project workspace frozen install — I isolated the peer constraint using the real published packages rather than installing the whole monorepo, and this change only realigns the declared dependency graph without modifying runtime code paths. The repo CI exercises the memory-lancedb suite and the package-manifest / extension-runtime-dependencies contract checks on this PR.

Limitations

This is a minimal dependency alignment for the current @lancedb/[email protected] peer contract. A future LanceDB release that supports Arrow 21 can relax this pin separately.

Dependency-graph guard

This PR intentionally changes the dependency graph (extensions/memory-lancedb/package.json dependencies, that package's npm-shrinkwrap.json, and the root pnpm-lock.yaml), so it is correctly held by the dependency-graph guard. The three files must change together: dropping the lockfile/shrinkwrap regeneration leaves apache-arrow mismatched against the manifest and a frozen install fails (ERR_PNPM_OUTDATED_LOCKFILE), so the lockfile changes are required, not incidental.

Per the guard, this needs an admin or @openclaw/openclaw-secops reviewer to inspect the generated graph delta and authorize the current head SHA with /allow-dependencies-change.

@sahibzada-allahyar
sahibzada-allahyar requested a review from a team as a code owner June 4, 2026 16:10
@github-actions github-actions Bot added the dependencies-changed PR changes dependency-related files label Jun 4, 2026
@github-actions

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Dependency Guard

This PR changes dependency-related files. Maintainers should confirm these changes are intentional.

Changed files:

  • extensions/memory-lancedb/npm-shrinkwrap.json
  • extensions/memory-lancedb/package.json
  • pnpm-lock.yaml

Maintainer follow-up:

  • Review whether the dependency changes are intentional.
  • Inspect resolved package deltas when lockfile, shrinkwrap, or workspace dependency policy changes are present.
  • Treat package-lock.json and npm-shrinkwrap.json diffs as security-review surfaces.
  • Run pnpm deps:changes:report -- --base-ref origin/main --markdown /tmp/dependency-changes.md --json /tmp/dependency-changes.json locally for detailed release-style evidence.

@openclaw-barnacle openclaw-barnacle Bot added extensions: memory-lancedb Extension: memory-lancedb triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. size: XS labels Jun 4, 2026
@github-actions

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Dependency graph changes are blocked

OpenClaw does not accept dependency graph changes through PRs unless a repository admin or security explicitly authorizes the current head SHA. Dependency updates are generated internally by maintainers so external PRs cannot change the resolved graph.

Detected dependency graph changes:

  • extensions/memory-lancedb/npm-shrinkwrap.json changed.
  • pnpm-lock.yaml changed.
  • extensions/memory-lancedb/package.json changed dependencies.

Auto-scrub was not attempted because this PR changes package manifest dependency graph fields:

  • extensions/memory-lancedb/package.json changed dependencies.

Dependency graph changes must be reviewed by security or handled by maintainers internally. Please remove lockfile changes manually if they are not needed.

To remove lockfile changes, restore them from the target branch:

git fetch origin
git checkout 'origin/main' -- 'extensions/memory-lancedb/npm-shrinkwrap.json' 'pnpm-lock.yaml'
git commit -m 'chore: remove dependency lockfile change'
git push

If this PR intentionally needs a dependency graph change, ask a repository admin or member of @openclaw/openclaw-secops to comment:

/allow-dependencies-change

The action will approve the current head SHA (f4fea32b3def76e19b3e3ab5a29ab3524521e2d0) when it reruns. A later push requires a fresh approval.

@sahibzada-allahyar

Copy link
Copy Markdown
Contributor Author

This dependency graph change is intentional for #90295: @lancedb/[email protected] declares apache-arrow peer range >=15.0.0 <=18.1.0, while the current memory-lancedb package pinned [email protected] and npm 11 rejects the install. This PR pins apache-arrow to 18.1.0 and regenerates the plugin shrinkwrap with the repo generator.\n\nLocal proof run on this head:\n- npm view @lancedb/[email protected] peerDependencies --json -> { "apache-arrow": ">=15.0.0 <=18.1.0" }\n- pnpm install --frozen-lockfile -> already up to date\n- node scripts/generate-npm-shrinkwrap.mjs --check --package-dir extensions/memory-lancedb -> current\n- npm install --prefix --ignore-scripts ./extensions/memory-lancedb --package-lock-only -> up to date\n- pnpm test extensions/memory-lancedb --reporter=dot -> 134 passed\n- node scripts/run-vitest.mjs src/plugins/contracts/package-manifest.contract.test.ts src/plugins/contracts/extension-runtime-dependencies.contract.test.ts --reporter=dot -> 430 passed\n\nDependency Guard requires a repo admin/security approval for this head SHA if maintainers want this dependency repair via external PR.

@clawsweeper

clawsweeper Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 30, 2026, 10:45 AM ET / 14:45 UTC.

Summary
The PR changes memory-lancedb's apache-arrow dependency from 21.1.0 to 18.1.0 and regenerates the plugin shrinkwrap plus root pnpm lockfile.

PR surface: Source +14, Config 0, Other +2. Total +16 across 3 files.

Reproducibility: yes. Source, shrinkwrap, root lockfile, latest published package metadata, and live LanceDB npm metadata show Arrow 21.1.0 outside LanceDB 0.30.0's accepted peer range; I did not run an install in this read-only review.

Review metrics: 1 noteworthy metric.

  • Dependency graph surfaces: 1 manifest dependency changed; 2 graph files regenerated. The manifest, plugin shrinkwrap, and root lockfile must be reviewed together because dropping any one leaves install metadata inconsistent.

Stored data model
Persistent data-model change detected: unknown-data-model-change: extensions/memory-lancedb/npm-shrinkwrap.json, vector/embedding metadata: extensions/memory-lancedb/npm-shrinkwrap.json. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #90295
Summary: This PR is the open fix candidate for the canonical memory-lancedb Arrow/LanceDB peer conflict issue; the later short report was already closed as a duplicate.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦐 gold shrimp
Proof: 🦞 diamond lobster
Patch quality: 🦐 gold shrimp
Result: needs maintainer review before merge.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Rebase against current main and regenerate the package graph files if they drift.
  • [P2] Have secops/admin inspect the resolved graph and authorize the exact refreshed head SHA, or apply the same fix from a trusted maintainer branch.

Risk before merge

  • [P1] The PR intentionally changes a package manifest, plugin shrinkwrap, and root lockfile from an external branch, so secops/admin review must authorize the exact resolved graph before merge.
  • [P1] GitHub reports the PR as dirty/conflicting; any rebase or regenerated graph will produce a new head SHA that needs fresh dependency-graph authorization.
  • [P1] Downgrading Arrow to the LanceDB-supported peer range changes transitive shrinkwrap packages, so install proof does not replace supply-chain review of the resolved graph.

Maintainer options:

  1. Refresh and authorize the exact graph (recommended)
    Rebase against current main, regenerate graph files if needed, inspect the dependency delta, then have secops/admin authorize the exact refreshed head SHA before merge.
  2. Apply the repair internally
    Maintainers can reproduce the same manifest, shrinkwrap, and lockfile change from a trusted branch and use that as the landing path.
  3. Pause the external graph change
    If maintainers do not want external dependency graph changes, keep this PR paused or close it only after an internal replacement is opened or merged.

Next step before merge

  • [P2] Manual review is needed because the remaining work is trusted dependency-graph authorization plus conflict refresh, not a safe autonomous code repair on this external branch.

Security
Needs attention: The diff is a legitimate dependency repair, but the external dependency graph and shrinkwrap changes need trusted resolved-graph review before merge.

Review details

Best possible solution:

Refresh or recreate the same dependency alignment from a trusted maintainer path, inspect the resolved graph, authorize the exact head SHA, then land and publish the repaired memory-lancedb package.

Do we have a high-confidence way to reproduce the issue?

Yes. Source, shrinkwrap, root lockfile, latest published package metadata, and live LanceDB npm metadata show Arrow 21.1.0 outside LanceDB 0.30.0's accepted peer range; I did not run an install in this read-only review.

Is this the best way to solve the issue?

Yes. Pinning Arrow to the top of LanceDB 0.30.0's accepted peer range is the narrowest package-contract repair; npm still reports LanceDB 0.30.0 as latest with the same peer range.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 6cb82eaab865.

Label changes

Label justifications:

  • P2: The PR fixes install/update failure for one official memory plugin while core OpenClaw remains usable with fallback memory.
  • merge-risk: 🚨 compatibility: Changing the memory-lancedb dependency graph affects fresh installs and upgrades for users of the official LanceDB memory plugin.
  • merge-risk: 🚨 other: The lockfile and shrinkwrap delta is a supply-chain review surface that green functional CI does not settle.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦞 diamond lobster and patch quality is 🦐 gold shrimp.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (live_output): The PR body includes structured terminal output showing live npm peer metadata plus npm 11 before/after resolution for Arrow 21.1.0 versus 18.1.0.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes structured terminal output showing live npm peer metadata plus npm 11 before/after resolution for Arrow 21.1.0 versus 18.1.0.
Evidence reviewed

PR surface:

Source +14, Config 0, Other +2. Total +16 across 3 files.

View PR surface stats
Area Files Added Removed Net
Source 1 66 52 +14
Tests 0 0 0 0
Docs 0 0 0 0
Config 1 1 1 0
Generated 0 0 0 0
Other 1 39 37 +2
Total 3 106 90 +16

Security concerns:

  • [medium] Review the external dependency graph before merge — extensions/memory-lancedb/package.json:12
    The PR changes a package dependency, root lockfile, and plugin shrinkwrap, including transitive package downgrades; repository policy routes this through secops/admin exact-SHA authorization.
    Confidence: 0.94
  • [medium] Re-authorize after conflict resolution
    GitHub reports the branch as dirty/conflicting, so any rebase or regenerated graph will change the head SHA and need fresh dependency-graph authorization.
    Confidence: 0.88

What I checked:

  • AGENTS policy applied: Root policy treats dependency-touching work as requiring direct dependency inspection, and lockfiles/shrinkwrap as security surfaces. (AGENTS.md:11, 6cb82eaab865)
  • Plugin dependency ownership: The extensions guide says plugin runtime dependencies belong in the owning plugin package and runtime peers should be declared/provided there. (extensions/AGENTS.md:27, 6cb82eaab865)
  • Current main still has the mismatch: Current main declares @lancedb/lancedb 0.30.0 together with apache-arrow 21.1.0 in memory-lancedb. (extensions/memory-lancedb/package.json:11, 6cb82eaab865)
  • Checked-in peer range conflicts: The memory-lancedb shrinkwrap root depends on apache-arrow 21.1.0 while the LanceDB entry peers apache-arrow >=15.0.0 <=18.1.0. (extensions/memory-lancedb/npm-shrinkwrap.json:47, 6cb82eaab865)
  • Live dependency contract: npm registry metadata for @lancedb/[email protected] and latest reports peerDependencies.apache-arrow as >=15.0.0 <=18.1.0.
  • Latest published plugin still affected: npm registry metadata for @openclaw/memory-lancedb@latest reports version 2026.6.10 with apache-arrow 21.1.0 and @lancedb/lancedb 0.30.0.

Likely related people:

  • steipete: Git metadata shows commit 48ccc50 moved memory-lancedb to the LanceDB 0.30.0 and Arrow 21.1.0 dependency pair now outside LanceDB's peer range. (role: introduced dependency pair; confidence: high; commits: 48ccc50282eb; files: extensions/memory-lancedb/package.json, extensions/memory-lancedb/npm-shrinkwrap.json, pnpm-lock.yaml)
  • vincentkoc: The latest release tag still ships the affected dependency pair, and release metadata ties v2026.6.10 to this contributor. (role: recent release contributor; confidence: medium; commits: aa69b12d0086; files: extensions/memory-lancedb/package.json)
  • amittell: Recent merged memory-lancedb runtime work is useful routing context for validating plugin behavior after the package graph repair lands. (role: recent adjacent runtime contributor; confidence: medium; commits: 945faf8e6798; files: extensions/memory-lancedb/index.ts, extensions/memory-lancedb/index.test.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. labels Jun 4, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. label Jun 4, 2026
@sahibzada-allahyar

Copy link
Copy Markdown
Contributor Author

Author-side follow-up: I do not think there is a code/test change to make here without undoing the fix. The dependency graph change is intentional for #90295, and ClawSweeper already marked proof and patch quality sufficient.

Remaining blocker appears to be repository policy: dependency graph changes from an external PR need repo admin or secops approval for the current head SHA, or maintainers can choose to apply the same graph change internally from a trusted branch.

I am leaving the branch unchanged pending that maintainer/security decision.

@clawsweeper clawsweeper Bot mentioned this pull request Jun 6, 2026
@openclaw-barnacle openclaw-barnacle Bot added proof: supplied External PR includes structured after-fix real behavior proof. and removed proof: sufficient ClawSweeper judged the real behavior proof convincing. labels Jun 6, 2026
@sahibzada-allahyar

Copy link
Copy Markdown
Contributor Author

Confirming this is an intentional dependency-graph change for #90295, and the proof gate is now green.

Check status

  • Real behavior proof ✅ — the PR body now carries the structured after-fix proof: the live npm peer-range query plus a real npm 11 before/after reproduction against the published @lancedb/[email protected] ([email protected]ERESOLVE, exit 1; [email protected] → resolves, exit 0).
  • dependency-guard ⛔ — the only remaining blocker, which by policy needs an admin / @openclaw/openclaw-secops authorization of the current head SHA.

Exact delta (small and mechanical)

File Change
extensions/memory-lancedb/package.json one line: apache-arrow 21.1.018.1.0
extensions/memory-lancedb/npm-shrinkwrap.json regenerated via scripts/generate-npm-shrinkwrap.mjs (+66/−52)
pnpm-lock.yaml regenerated to match (+39/−37)

The three files must change together: dropping the lockfile/shrinkwrap regeneration leaves apache-arrow mismatched against the manifest, and a frozen install then fails with ERR_PNPM_OUTDATED_LOCKFILE. So the lockfile changes are required, not incidental.

To inspect the full resolved-graph delta:

pnpm deps:changes:report -- --base-ref origin/main --markdown /tmp/dependency-changes.md --json /tmp/dependency-changes.json

Could an admin or @openclaw/openclaw-secops member review the delta and, if acceptable, comment /allow-dependencies-change to authorize head f4fea32? If you'd rather keep dependency-graph changes internal, I'm happy to have it regenerated from a maintainer branch instead.

@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 6, 2026
@allenhurff

Copy link
Copy Markdown
Contributor

@steipete

Adding live user-impact context from #90295.

This is still affecting real installs after the 2026.6.5 update. The latest published @openclaw/[email protected] still declares:

But @lancedb/[email protected] still peers apache-arrow >=15.0.0 <=18.1.0, so normal npm resolution is still broken.

I have now had to keep memory-lancedb pinned back at 2026.5.28 across multiple OpenClaw updates to keep LanceDB memory working. That workaround is functional, but this is a core part of the memory stack and it is pretty painful to rediscover/reapply during updates.

It also looks like this PR is not really waiting on the author anymore. The author has already confirmed the dependency graph change is intentional and that the remaining blocker is maintainer/security authorization for the dependency change, or for maintainers to apply the same fix from an internal branch.

Could a maintainer either authorize this exact dependency fix or apply the equivalent trusted-branch fix and publish a repaired memory-lancedb package? The current published package is still broken for standard npm installs.

@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jun 19, 2026
@sahibzada-allahyar

Copy link
Copy Markdown
Contributor Author

This PR only realigns memory-lancedb's apache-arrow to 18.1.0 to match the version @lancedb/[email protected] actually resolves against (21.x drifts the peer and breaks the native Arrow FFI at runtime). The lockfile/shrinkwrap deltas are the mechanical result of that single version pin.

The dependency-guard check is the policy gate for any dependency-file change, so this needs a maintainer/secops /allow-dependencies-change to proceed. Could a maintainer apply it? Happy to rebase the lockfiles onto current main if the tree has drifted since.

@steipete

Copy link
Copy Markdown
Contributor

Thanks @sahibzada-allahyar. Your diagnosis and dependency-alignment patch were correct.

Closing as superseded by #99118 and commit 968aa51. Current main now pins apache-arrow to 18.1.0, regenerates the memory-lancedb shrinkwrap, and carries the matching root lockfile resolution. The canonical issue #90295 closed when that trusted maintainer replacement merged.

The currently published npm latest package still carries the old dependency pair as of today, so release publication remains follow-through work; it is no longer implementation work for this PR. Thanks for supplying the correct fix and proof.

@steipete steipete closed this Jul 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies-changed PR changes dependency-related files extensions: memory-lancedb Extension: memory-lancedb merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. size: XS status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

memory-lancedb 2026.6.1 fails to install: apache-arrow 21.1.0 conflicts with @lancedb/lancedb 0.30.0 peer range

3 participants