fix(plugin-sdk): guard legacy dedupe JSON parse against malformed files by Pick-cat · Pull Request #98125 · openclaw/openclaw

Pick-cat · 2026-06-30T11:20:41Z

What Problem This Solves

listPersistentDedupeLegacyJsonFileEntries (used during openclaw doctor legacy dedupe migration) calls JSON.parse on retired JSON cache files without catching parse errors. If a legacy dedupe file is partially written, truncated, or otherwise malformed, the entire doctor repair flow throws a SyntaxError instead of treating the file as empty.

Changes

src/plugin-sdk/persistent-dedupe.ts: parseLegacyDedupeData now wraps JSON.parse in a try/catch. Malformed JSON is treated the same way as {} or [] — returning { data: {}, invalidCount: 0 } — so the legacy migration continues instead of crashing.
src/plugin-sdk/memory-host-events.test.ts: added a regression test that writes {not valid json to a legacy dedupe file and asserts listPersistentDedupeLegacyJsonFileEntries resolves to an empty array.

Real behavior proof

Behavior addressed: A malformed legacy dedupe JSON file must not crash the migration entry reader; it must be treated as empty while well-formed files continue to load normally.
Real environment tested: Local Node v22 workspace, using the exported production function listPersistentDedupeLegacyJsonFileEntries against a real temporary file on disk.
Exact steps: node --import tsx /tmp/proof-persistent-dedupe.mts writes a malformed legacy dedupe file, calls the production reader, and prints the result. The same script also verifies a well-formed file still returns the expected entry.
Observed result after fix: malformed file → RESULT: ok, entries=0; well-formed file → RESULT: ok, entries=1. Before the fix the malformed file threw SyntaxError: Expected property name or '}' in JSON at position 1.
What was not tested: The full openclaw doctor --fix end-to-end flow was not run; the proof focuses on the exact entry-point function that doctor uses, and the in-repo Vitest regression locks the behavior.

Evidence

Terminal proof (real production function, real malformed file on disk):

$ node --import tsx /tmp/proof-persistent-dedupe.mts
malformed: RESULT: ok, entries=0
valid:     RESULT: ok, entries=1

Negative control (temporary revert to origin/main):

$ git checkout origin/main -- src/plugin-sdk/persistent-dedupe.ts
$ node --import tsx /tmp/proof-persistent-dedupe.mts
malformed: RESULT: error, Expected property name or '}' in JSON at position 1 (line 1 column 2)
valid:     RESULT: ok, entries=1
$ git checkout HEAD -- src/plugin-sdk/persistent-dedupe.ts

Focused Vitest:

$ node scripts/run-vitest.mjs src/plugin-sdk/memory-host-events.test.ts --run

Test Files  1 passed (1)
      Tests  13 passed (13)
   Duration  1.37s

Type checks:

$ pnpm tsgo:core      # exit 0
$ pnpm tsgo:test:src  # exit 0

clawsweeper · 2026-06-30T11:21:48Z

Codex review: needs maintainer review before merge. Reviewed June 30, 2026, 7:37 AM ET / 11:37 UTC.

Summary
The PR wraps legacy persistent dedupe JSON parsing in a catch path and adds a malformed-file regression test.

PR surface: Source +5, Tests +14. Total +19 across 2 files.

Reproducibility: yes. Current main reads the malformed legacy file and calls JSON.parse without a catch, and the PR body includes terminal negative-control proof showing the SyntaxError path.

Review metrics: none identified.

Stored data model
Persistent data-model change detected: persistent cache schema: src/plugin-sdk/memory-host-events.test.ts, serialized state: src/plugin-sdk/memory-host-events.test.ts, serialized state: src/plugin-sdk/persistent-dedupe.ts, vector/embedding metadata: src/plugin-sdk/memory-host-events.test.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

none.

Next step before merge

No ClawSweeper repair lane is needed; the PR has adequate proof and no blocking findings, so the remaining action is ordinary maintainer review plus required checks.

Security
Cleared: The diff only changes local JSON parsing and a focused test, with no dependency, workflow, package, secret, permission, or code-execution surface change.

Review details

Best possible solution:

Merge the narrow parser guard and regression test after required maintainer checks, keeping active SQLite dedupe behavior unchanged while hardening retired JSON migration input.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main reads the malformed legacy file and calls JSON.parse without a catch, and the PR body includes terminal negative-control proof showing the SyntaxError path.

Is this the best way to solve the issue?

Yes. The shared parser is the right fix point because it covers direct listing and migration callers without duplicating malformed-file handling in each plugin doctor path.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 6cb82eaab865.

Label changes

Label justifications:

P2: This is a normal-priority bug fix for a doctor legacy-state migration path that can fail on corrupted retired dedupe cache files.
rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): Sufficient terminal proof is in the PR body: it exercises the production reader on a real malformed file, includes a current-main negative control, and reports focused Vitest and typecheck results.
proof: sufficient: Contributor real behavior proof is sufficient. Sufficient terminal proof is in the PR body: it exercises the production reader on a real malformed file, includes a current-main negative control, and reports focused Vitest and typecheck results.

Evidence reviewed

PR surface:

Source +5, Tests +14. Total +19 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	6	1	+5
Tests	1	14	0	+14
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	20	1	+19

What I checked:

Root repository policy read: Root AGENTS.md was read fully and its PR-review guidance made scoped SDK policy, callers, sibling migration behavior, current-main behavior, proof, and history relevant to the verdict. (AGENTS.md:1, 6cb82eaab865)
Scoped SDK policy read: src/plugin-sdk/AGENTS.md identifies this directory as the public core/plugin contract, so the review treated the helper as compatibility-sensitive but found no public API shape change. (src/plugin-sdk/AGENTS.md:1, 6cb82eaab865)
Current main behavior: Current main reads the legacy file and calls JSON.parse without a catch in parseLegacyDedupeData, so malformed retired JSON can reject before entries are returned. (src/plugin-sdk/persistent-dedupe.ts:329, 6cb82eaab865)
Affected callers: The helper feeds real migration paths: Nextcloud Talk doctor calls migratePersistentDedupeLegacyJsonFile, and Telegram legacy state migration calls listPersistentDedupeLegacyJsonFileEntries. (extensions/nextcloud-talk/src/doctor.ts:81, 6cb82eaab865)
Sibling behavior: Feishu's legacy dedupe migration already catches malformed legacy JSON and treats it as empty, which supports the PR's behavior as consistent migration-boundary handling rather than a new runtime fallback. (extensions/feishu/src/dedup-migrations.ts:20, 6cb82eaab865)
PR diff: The PR catches JSON.parse failures in parseLegacyDedupeData and adds a test asserting malformed legacy JSON resolves to an empty entry list. (src/plugin-sdk/persistent-dedupe.ts:326, ab1a79761399)

Likely related people:

Pick-cat: Current-main blame for parseLegacyDedupeData, the migration helper, and the Telegram/Nextcloud callers points to the merged work that introduced this legacy dedupe migration surface. (role: introduced current helper and recent area contributor; confidence: high; commits: eb5fb2aa69f4, 3a3dd59e1803; files: src/plugin-sdk/persistent-dedupe.ts, src/plugin-sdk/memory-host-events.test.ts, extensions/telegram/src/state-migrations.ts)
vincentkoc: Live PR metadata shows this person merged the prior PR whose current-main merge commit carries the affected helper surface. (role: merger; confidence: medium; commits: eb5fb2aa69f4, 3a3dd59e1803; files: src/plugin-sdk/persistent-dedupe.ts, extensions/telegram/src/state-migrations.ts, extensions/nextcloud-talk/src/doctor.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Pick-cat · 2026-06-30T11:57:58Z

@clawsweeper re-review

clawsweeper · 2026-06-30T11:58:01Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

vincentkoc · 2026-06-30T16:56:26Z

Merged via squash.

Prepared head SHA: ab1a79761399b0b19712be24e2f93183c3635557
Landed commit: 169acd1e4ed4da06cb046df0c8fd2d61b2b3d700

…es (openclaw#98125)

…es (openclaw#98125) (cherry picked from commit 169acd1)

fix(plugin-sdk): guard legacy dedupe JSON parse against malformed files

ab1a797

openclaw-barnacle Bot added the size: XS label Jun 30, 2026

clawsweeper Bot mentioned this pull request Jun 30, 2026

fix(infra): bound jsonl-socket response buffer to prevent OOM #98130

Merged

vincentkoc merged commit 169acd1 into openclaw:main Jun 30, 2026
156 of 164 checks passed

github-actions Bot mentioned this pull request Jun 30, 2026

📡 Upstream Digest — 2026-06-30 17:46 UTC curtismercier/openclaw-mods#1177

Open

This was referenced Jul 1, 2026

fix(telegram): guard ingress worker JSON.parse against malformed Bot API bodies #98372

Closed

fix(memory-core): guard qmd mcporter JSON.parse against non-JSON stdout #98381

Merged

github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 1, 2026

fix(plugin-sdk): guard legacy dedupe JSON parse against malformed fil…

5ce7aaa

…es (openclaw#98125)

QiuYuang pushed a commit to QiuYuang/openclaw that referenced this pull request Jul 1, 2026

fix(plugin-sdk): guard legacy dedupe JSON parse against malformed fil…

71137cc

…es (openclaw#98125)

chenyangjun-xy pushed a commit to chenyangjun-xy/openclaw that referenced this pull request Jul 1, 2026

fix(plugin-sdk): guard legacy dedupe JSON parse against malformed fil…

dbb03eb

…es (openclaw#98125)

miorbnli mentioned this pull request Jul 2, 2026

fix(codex): guard sandbox http/request JSON.parse against non-JSON stdout #98881

Closed

chenyangjun-xy pushed a commit to chenyangjun-xy/openclaw that referenced this pull request Jul 3, 2026

fix(plugin-sdk): guard legacy dedupe JSON parse against malformed fil…

8ce4699

…es (openclaw#98125)

Rorqualx pushed a commit to Rorqualx/cortex that referenced this pull request Jul 6, 2026

fix(plugin-sdk): guard legacy dedupe JSON parse against malformed fil…

d0751de

…es (openclaw#98125) (cherry picked from commit 169acd1)

Rorqualx pushed a commit to Rorqualx/cortex that referenced this pull request Jul 8, 2026

fix(plugin-sdk): guard legacy dedupe JSON parse against malformed fil…

4856eb4

…es (openclaw#98125) (cherry picked from commit 169acd1)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(plugin-sdk): guard legacy dedupe JSON parse against malformed files#98125

fix(plugin-sdk): guard legacy dedupe JSON parse against malformed files#98125
vincentkoc merged 1 commit into
openclaw:mainfrom
Pick-cat:fix/persistent-dedupe-malformed-json

Pick-cat commented Jun 30, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jun 30, 2026 •

edited

Loading

Uh oh!

Pick-cat commented Jun 30, 2026

Uh oh!

clawsweeper Bot commented Jun 30, 2026 •

edited

Loading

Uh oh!

Uh oh!

vincentkoc commented Jun 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

Pick-cat commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Changes

Real behavior proof

Evidence

Uh oh!

clawsweeper Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Pick-cat commented Jun 30, 2026

Uh oh!

clawsweeper Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

vincentkoc commented Jun 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Pick-cat commented Jun 30, 2026 •

edited

Loading

clawsweeper Bot commented Jun 30, 2026 •

edited

Loading

clawsweeper Bot commented Jun 30, 2026 •

edited

Loading