Refactor external plugin catalog toward feeds by giodl73-repo · Pull Request #95846 · openclaw/openclaw

giodl73-repo · 2026-06-22T19:55:25Z

Hosted Marketplace Feed Stack

This is the first PR in the hosted marketplace feed stack. The stack should land in order, with maintainer review at each step before the later PRs wire hosted marketplace data into broader user flows.

Refactor external plugin catalog toward feeds #95846: refactor the bundled external plugin catalog toward the feed-shaped data model.
Add hosted external catalog feed loader #95868: add the lazy hosted marketplace feed loader for the configured public feed URL, with guarded HTTPS fetch, checksum, schema, size, timeout, and bundled fallback behavior.
Add hosted catalog snapshot fallback #95877: add accepted hosted snapshot fallback so a failed or unchanged hosted fetch can reuse the last valid feed payload.
Persist hosted catalog snapshots in state #95964: persist hosted marketplace feed snapshots in OpenClaw state.
Add hosted catalog source profile validation #95969: add marketplace source profile validation for hosted feed install candidates.
Add hosted catalog config profiles #95981: add configurable marketplaces.feeds and marketplaces.sources profiles.
Add marketplace feed refresh command #96155: add openclaw plugins marketplace refresh as the explicit manual snapshot refresh path.
Add marketplace feed entries command #96158: add openclaw plugins marketplace entries as the explicit manual read path over hosted, snapshot, or bundled fallback entries.
Add marketplace feed telemetry #96194: add marketplace feed telemetry.

The broader follow-up after this stack is to decide whether and how marketplace entries should feed install/search/startup behavior, add automatic conditional refresh using ETag / Last-Modified, or move to ClawHub producer-side validation for /v1/feeds/plugins and /v1/feeds/skills.

Journey Context

This is step 1 of the hosted OpenClaw feed path from RFC 19. This PR does the smallest safe prep: it makes the bundled official external plugin catalog look like the feed document we want to host later, while preserving current local behavior.

What This PR Does

Adds feed metadata around scripts/lib/official-external-plugin-catalog.json without changing the existing entries payload.
Adds parser/type support for the feed-shaped bundled catalog.
Preserves legacy local catalog parsing for existing unversioned catalog shapes.
Fails closed for unsupported versioned feed wrappers so future schema versions do not silently parse as legacy catalogs.

What This PR Intentionally Does Not Do

This PR does not fetch from register.openclaw.ai, add auth, add source profiles, add signed envelopes, add regional selection, add snapshot persistence, change search behavior, or move skills into feeds. Those are separate reviewable slices.

How To Review

Review this as a mechanical shape change plus compatibility guard. Existing callers should still get the same official external plugin entries from the bundled catalog, while the bundled file becomes ready to serve as the local fallback for a hosted feed.

Validation

pnpm exec oxfmt --check src/plugins/official-external-plugin-catalog.ts src/plugins/official-external-plugin-catalog.test.ts
pnpm exec oxlint --deny-warnings src/plugins/official-external-plugin-catalog.ts src/plugins/official-external-plugin-catalog.test.ts
JSON parse of scripts/lib/official-external-plugin-catalog.json
git diff --check
CI=1 timeout 180s node scripts/test-projects.mjs src/plugins/official-external-plugin-catalog.test.ts (17 tests at initial PR authoring)

Real Behavior Proof

Behavior or issue addressed:
The feed-shaped bundled fallback is still loaded by existing runtime catalog consumers after the catalog wrapper refactor.

Real environment tested:
Windows checkout C:\tmp\openclaw-feeds-proof\pr1 at 44c0884c3edc, source harness via node --import tsx, no network services.

Exact steps or command run after this patch:

Imported src/plugins/official-external-plugin-catalog.ts.
Imported src/plugins/web-search-install-catalog.ts.
Listed official external catalog entries and web-search consumer candidates from the bundled fallback.

Evidence after fix:

{
  "head": "44c0884c3edc",
  "bundledFeedAccepted": true,
  "officialEntryCount": 74,
  "firstOfficialIds": ["wecom-openclaw-plugin", "openclaw-plugin-yuanbao", "openclaw-weixin", "openclaw-zaloclawbot"],
  "webSearchConsumerCount": 9,
  "webSearchConsumerSample": [
    { "pluginId": "brave", "provider": "brave", "npmSpec": "@openclaw/brave-plugin" },
    { "pluginId": "exa", "provider": "exa", "npmSpec": "@openclaw/exa-plugin" },
    { "pluginId": "firecrawl", "provider": "firecrawl", "npmSpec": "@openclaw/firecrawl-plugin" }
  ]
}

Observed result after fix:
The bundled feed wrapper validates, the existing official catalog returns 74 entries, and a real catalog consumer still resolves web-search install candidates from that feed-shaped fallback.

What was not tested:
Hosted network fetching is intentionally not part of this PR. It is covered by the next hosted-fetch PR.

clawsweeper · 2026-06-22T20:26:53Z

Codex review: needs maintainer review before merge. Reviewed June 27, 2026, 10:39 AM ET / 14:39 UTC.

Summary
The PR wraps the bundled official external plugin catalog in feed metadata and updates parser/tests to accept that wrapper while preserving legacy unversioned catalog shapes.

PR surface: Source +44, Tests +51, Other +5. Total +100 across 3 files.

Reproducibility: not applicable. as a bug reproduction: this is a preparatory marketplace feed refactor. Source inspection and the PR body proof show the intended feed-wrapper path and existing catalog consumer path.

Review metrics: 1 noteworthy metric.

Feed wrapper metadata: 5 added fields. The bundled fallback gains version, identity, timestamp, sequence, and description fields that later hosted-feed consumers will treat as contract surface.

Stored data model
Persistent data-model change detected: database schema: scripts/lib/official-external-plugin-catalog.json, database schema: src/plugins/official-external-plugin-catalog.test.ts, database schema: src/plugins/official-external-plugin-catalog.ts. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: canonical
Canonical: #95846
Summary: This PR is the first feed-wrapper slice in an open hosted marketplace feed stack; the other stack PRs are adjacent dependent slices, not duplicates or superseding replacements.

Members:

adjacent_distinct: Add hosted external catalog feed loader #95868 - Open follow-up PR adds the hosted external catalog feed loader that depends on this feed-shaped fallback.
adjacent_distinct: Add hosted catalog snapshot fallback #95877 - Open follow-up PR adds hosted catalog snapshot fallback semantics on top of the loader stack.
adjacent_distinct: Persist hosted catalog snapshots in state #95964 - Open follow-up PR persists hosted catalog snapshots after the snapshot fallback contract.
adjacent_distinct: Add hosted catalog source profile validation #95969 - Open follow-up PR validates hosted catalog source profiles for feed install candidates.
adjacent_distinct: Add hosted catalog config profiles #95981 - Open follow-up PR adds configurable marketplace feed and source profiles.
3 more in the report.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

[P1] Get maintainer confirmation that schemaVersion 1, fail-closed unsupported-version handling, and stack order are the intended marketplace feed contract.

Risk before merge

[P1] The schemaVersion 1 wrapper and unsupported-version fail-closed behavior become compatibility surface for later hosted marketplace feed PRs, so maintainers need to confirm that contract before merge.
[P1] A parser or wrapper mismatch could hide official external entries from install, doctor, onboarding, provider setup, and web-search setup paths even though the current head source and tests look aligned.

Maintainer options:

Confirm the feed contract (recommended)
Maintainers can explicitly accept schemaVersion 1, required wrapper fields, and fail-closed unsupported versions as the base contract for the hosted marketplace feed stack.
Pause until the producer schema settles
If the ClawHub feed schema is still moving, keep this PR open or replace it with the final wrapper shape before dependent PRs build on it.
Keep this as the isolated first slice
If accepted, merge this slice before the hosted fetch, config, storage, and CLI follow-ups so parser compatibility stays reviewable independently.

Next step before merge

[P2] The remaining action is maintainer product/API confirmation for a compatibility-sensitive feed contract, not a narrow automated code repair.

Security
Cleared: The diff changes bundled JSON metadata, parser logic, and tests only; it does not add dependencies, workflows, secrets handling, downloads, install scripts, or package-resolution execution.

Review details

Best possible solution:

Land this parser/control-plane prep only after maintainers confirm the schemaVersion 1 wrapper, fail-closed behavior, and stack order; keep hosted fetching, persistence, config, and CLI activation in the follow-up PRs.

Do we have a high-confidence way to reproduce the issue?

Not applicable as a bug reproduction: this is a preparatory marketplace feed refactor. Source inspection and the PR body proof show the intended feed-wrapper path and existing catalog consumer path.

Is this the best way to solve the issue?

Yes for the implementation layer: the parser/control-plane module is the narrow place to accept a bundled feed wrapper without changing callers. The remaining question is maintainer acceptance of the schema contract and rollout order, not a code defect.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against f4fa10c2c5ac.

Label changes

Label justifications:

P3: This is preparatory hosted-feed work with limited immediate user impact, despite needing compatibility-aware maintainer confirmation before merge.
merge-risk: 🚨 compatibility: The PR changes the packaged official external plugin catalog fallback and parser used by install, doctor, onboarding, provider, and web-search catalog paths.
rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body includes after-patch live output showing the feed wrapper accepted, official entries loaded, and a real web-search catalog consumer resolving candidates; screenshots are not needed for this non-visual parser path.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-patch live output showing the feed wrapper accepted, official entries loaded, and a real web-search catalog consumer resolving candidates; screenshots are not needed for this non-visual parser path.

Evidence reviewed

PR surface:

Source +44, Tests +51, Other +5. Total +100 across 3 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	46	2	+44
Tests	1	51	0	+51
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	1	5	0	+5
Total	3	102	2	+100

What I checked:

Repository policy applied: Root policy was read fully; plugin catalog fallback parsing and future marketplace feed contracts are compatibility-sensitive under the repo review guidance. (AGENTS.md:31, f4fa10c2c5ac)
Scoped plugin policy applied: The scoped plugin guide was read fully and keeps discovery/catalog control-plane behavior in the plugin boundary with stable metadata contracts. (src/plugins/AGENTS.md:3, f4fa10c2c5ac)
Current main behavior: Current main parses arrays or legacy entries/packages/plugins objects directly and has no schemaVersion-aware feed wrapper or unsupported-version guard, so the PR is not obsolete. (src/plugins/official-external-plugin-catalog.ts:100, f4fa10c2c5ac)
PR feed parser: The PR head adds a feed type, accepts only schemaVersion 1 with required wrapper fields, parses feed entries through the same record filter, and rejects other versioned wrappers. (src/plugins/official-external-plugin-catalog.ts:87, 82d05bdc461e)
Bundled catalog wrapper: The PR head adds schemaVersion, id, generatedAt, sequence, and description metadata before the existing entries array. (scripts/lib/official-external-plugin-catalog.json:2, 82d05bdc461e)
Regression coverage: The PR adds tests for the bundled feed shape, malformed wrappers, unsupported versioned wrappers, and legacy unversioned catalog parsing. (src/plugins/official-external-plugin-catalog.test.ts:26, 82d05bdc461e)

Likely related people:

vincentkoc: Recent GitHub history shows multiple current-main changes to the official external catalog parser and bundled catalog JSON used by this PR. (role: recent external catalog contributor; confidence: high; commits: f2eca9439189, 0710eebec2b0, 21728777dfa0; files: src/plugins/official-external-plugin-catalog.ts, scripts/lib/official-external-plugin-catalog.json)
steipete: Recent history shows normalization/helper refactors on the catalog parser and adjacent web-search install catalog surface. (role: shared parser/helper refactor contributor; confidence: medium; commits: 00d8d7ead059, 77d9ac30bb8d, f6049db20fe4; files: src/plugins/official-external-plugin-catalog.ts, src/plugins/web-search-install-catalog.ts)
Patrick-Erichsen: This person approved the PR and recently authored a current-main external provider catalog change touching the same official external catalog area. (role: recent reviewer and adjacent contributor; confidence: medium; commits: 1b19c790bfdd; files: src/plugins/official-external-plugin-catalog.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

giodl73-repo · 2026-06-22T20:47:31Z

@clawsweeper re-review

clawsweeper · 2026-06-22T20:47:34Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

giodl73-repo · 2026-06-24T13:59:11Z

@clawsweeper re-review

giodl73-repo · 2026-06-26T05:34:35Z

@clawsweeper re-review

giodl73-repo · 2026-06-27T14:41:47Z

Merged via squash.

Prepared head SHA: 82d05bdc461eab94db7e0ad4e7d99650daf84815
Merge commit: 4010b81a77f40c17ebfc55a6b47571dc1ef92902

Thanks @giodl73-repo!

@giodl73-repo

Merged via squash. Prepared head SHA: 82d05bd Co-authored-by: giodl73-repo <[email protected]> Co-authored-by: giodl73-repo <[email protected]> Reviewed-by: @giodl73-repo

@giodl73-repo

Merged via squash. Prepared head SHA: 82d05bd Co-authored-by: giodl73-repo <[email protected]> Co-authored-by: giodl73-repo <[email protected]> Reviewed-by: @giodl73-repo

@giodl73-repo

Merged via squash. Prepared head SHA: 82d05bd Co-authored-by: giodl73-repo <[email protected]> Co-authored-by: giodl73-repo <[email protected]> Reviewed-by: @giodl73-repo

@giodl73-repo

Merged via squash. Prepared head SHA: 82d05bd Co-authored-by: giodl73-repo <[email protected]> Co-authored-by: giodl73-repo <[email protected]> Reviewed-by: @giodl73-repo

openclaw-barnacle Bot added scripts Repository scripts size: S maintainer Maintainer-authored PR labels Jun 22, 2026

Patrick-Erichsen approved these changes Jun 23, 2026

View reviewed changes

vincentkoc mentioned this pull request Jun 23, 2026

feat(catalog): publish hosted OpenClaw plugin feed openclaw/clawhub#2805

Merged

This was referenced Jun 23, 2026

Add marketplace feed refresh command #96155

Merged

Add marketplace feed entries command #96158

Merged

clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. labels Jun 24, 2026

clawsweeper Bot added status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Jun 27, 2026

giodl73-repo merged commit 4010b81 into openclaw:main Jun 27, 2026
111 of 116 checks passed

giodl73-repo deleted the feeds-hosted-catalog-prep branch June 27, 2026 14:41

github-actions Bot mentioned this pull request Jun 27, 2026

📡 Upstream Digest — 2026-06-27 15:09 UTC curtismercier/openclaw-mods#1159

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Refactor external plugin catalog toward feeds#95846

Refactor external plugin catalog toward feeds#95846
giodl73-repo merged 1 commit into
openclaw:mainfrom
giodl73-repo:feeds-hosted-catalog-prep

giodl73-repo commented Jun 22, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jun 22, 2026 •

edited

Loading

Uh oh!

giodl73-repo commented Jun 22, 2026

Uh oh!

clawsweeper Bot commented Jun 22, 2026 •

edited

Loading

Uh oh!

giodl73-repo commented Jun 24, 2026

Uh oh!

giodl73-repo commented Jun 26, 2026

Uh oh!

Uh oh!

giodl73-repo commented Jun 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

giodl73-repo commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Hosted Marketplace Feed Stack

Journey Context

What This PR Does

What This PR Intentionally Does Not Do

How To Review

Validation

Real Behavior Proof

Uh oh!

clawsweeper Bot commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

giodl73-repo commented Jun 22, 2026

Uh oh!

clawsweeper Bot commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

giodl73-repo commented Jun 24, 2026

Uh oh!

giodl73-repo commented Jun 26, 2026

Uh oh!

Uh oh!

giodl73-repo commented Jun 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

giodl73-repo commented Jun 22, 2026 •

edited

Loading

clawsweeper Bot commented Jun 22, 2026 •

edited

Loading

clawsweeper Bot commented Jun 22, 2026 •

edited

Loading