Add marketplace feed refresh command#96155
Conversation
c41431f to
76baf5a
Compare
|
Codex review: needs maintainer review before merge. Reviewed June 28, 2026, 1:07 PM ET / 17:07 UTC. Summary PR surface: Source +236, Tests +478, Docs +19. Total +733 across 7 files. Reproducibility: not applicable. This PR adds a new manual CLI capability rather than fixing an existing current-main bug. Current main and v2026.6.10 lack the refresh command, while the PR body includes live-output proof for the new path. Review metrics: 1 noteworthy metric.
Stored data model Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Risk before merge
Maintainer options:
Next step before merge
Security Review detailsBest possible solution: Land this slice only after maintainers accept the command name, flags, JSON shape, fallback modes, URL restrictions, and snapshot-write failure behavior as the supported manual hosted-feed refresh API. Do we have a high-confidence way to reproduce the issue? Not applicable; this PR adds a new manual CLI capability rather than fixing an existing current-main bug. Current main and v2026.6.10 lack the refresh command, while the PR body includes live-output proof for the new path. Is this the best way to solve the issue? Yes as an implementation slice: it reuses the existing hosted loader, source-profile checks, SSRF guard, and snapshot store instead of adding a parallel fetch path. The remaining question is maintainer acceptance of the public CLI contract and fallback UX. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 891096926e27. Label changesLabel justifications:
Evidence reviewedPR surface: Source +236, Tests +478, Docs +19. Total +733 across 7 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
|
fdd7f78 to
05a3e13
Compare
76baf5a to
9dcb7b7
Compare
05a3e13 to
d1f3927
Compare
9dcb7b7 to
0e4486b
Compare
|
Addressed the marketplace feed URL output concern in this slice: explicit feed URLs with userinfo are rejected before fetch, and marketplace refresh JSON/text output now redacts query strings, fragments, username, and password from emitted feed URLs and fallback errors. Added focused regression coverage. @clawsweeper re-review |
d1f3927 to
c7565c0
Compare
0e4486b to
15b033a
Compare
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
c7565c0 to
da81650
Compare
15b033a to
82c4042
Compare
b93aa0e to
219aa1d
Compare
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
219aa1d to
034ab82
Compare
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
034ab82 to
1614070
Compare
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
Merged via squash.
Thanks @giodl73-repo! |
Merged via squash. Prepared head SHA: 1614070 Co-authored-by: giodl73-repo <[email protected]> Co-authored-by: giodl73-repo <[email protected]> Reviewed-by: @giodl73-repo
Merged via squash. Prepared head SHA: 1614070 Co-authored-by: giodl73-repo <[email protected]> Co-authored-by: giodl73-repo <[email protected]> Reviewed-by: @giodl73-repo
Merged via squash. Prepared head SHA: 1614070 Co-authored-by: giodl73-repo <[email protected]> Co-authored-by: giodl73-repo <[email protected]> Reviewed-by: @giodl73-repo
Merged via squash. Prepared head SHA: 1614070 Co-authored-by: giodl73-repo <[email protected]> Co-authored-by: giodl73-repo <[email protected]> Reviewed-by: @giodl73-repo
Hosted Marketplace Feed Stack
This is PR7 in the hosted marketplace feed stack. The stack should land in order, with maintainer review at each step before the later PRs wire hosted marketplace data into broader user flows.
marketplaces.feedsandmarketplaces.sourcesprofiles.openclaw plugins marketplace refreshas the explicit manual snapshot refresh path.openclaw plugins marketplace entriesas the explicit manual read path over hosted, snapshot, or bundled fallback entries.The broader follow-up after this stack is to decide whether and how marketplace entries should feed install/search/startup behavior, add automatic conditional refresh using
ETag/Last-Modified, or move to ClawHub producer-side validation for/v1/feeds/pluginsand/v1/feeds/skills.Summary
Adds
openclaw plugins marketplace refresh, a manual activation slice for the hosted marketplace feed stack. The command readsconfig.marketplaces, calls the configured hosted external plugin catalog loader, writes the accepted snapshot, and reports whether the accepted catalog came from the hosted feed, the last-known-good snapshot, or bundled fallback.This gives maintainers a safe way to exercise the live
https://clawhub.ai/v1/feeds/pluginspath and any locally configuredmarketplaces.feedsprofile without changing startup, onboarding, doctor, install, search, or provider-repair behavior yet.What Changed
plugins marketplace refreshwith:--feed-profile <name>for a configured marketplace feed profile.--feed-url <url>for the existing guarded explicit hosted URL path.--expected-sha256 <hash>for checksum-pinned refresh checks.--jsonfor automation-friendly source/feed/metadata output.sourceReffor non-default direct feed URLs.plugins marketplace refresh.Validation
pnpm exec oxfmt --check src/cli/plugins-cli.runtime.ts src/cli/plugins-cli.marketplace-refresh.test.ts src/plugins/official-external-plugin-catalog.ts src/plugins/official-external-plugin-catalog.test.ts docs/cli/plugins.mdpnpm exec oxlint --deny-warnings src/cli/plugins-cli.runtime.ts src/cli/plugins-cli.marketplace-refresh.test.ts src/plugins/official-external-plugin-catalog.ts src/plugins/official-external-plugin-catalog.test.tsgit diff --check$env:CI='1'; node scripts/test-projects.mjs src/cli/plugins-cli.lazy.test.ts src/cli/plugins-cli.marketplace-refresh.test.ts src/plugins/official-external-plugin-catalog.test.ts(8 CLI refresh/lazy tests and 51 catalog tests passed)pnpm docs:check-mdxpnpm tsgo:corecodex review --base origin/main(no actionable correctness issues)Note:
pnpm format:docs:checkstill fails on this Windows host before checking this file because the repo-wide docs formatter builds anoxfmtcommand line that exceeds the Windows command-line length limit. The targetedoxfmt --check docs/cli/plugins.mdandpnpm docs:check-mdxpassed.Real behavior proof
Behavior or issue addressed:
openclaw plugins marketplace refreshis the first manual activation path for hosted marketplace feeds. It accepts a fresh hosted payload and writes the accepted snapshot without enabling install/search/startup consumption.Real environment tested:
Windows checkout
C:\src\openclaw-prflow\.worktrees\pr-96155at pushed PR7 head1614070e92(runtime proof path unchanged from034ab82b95; current head adds docs-only refresh-behavior clarification), source CLI runtime harness vianode --import tsx, isolated state dirC:\tmp\openclaw-feeds-proof\pr96155-refresh-034ab82, live HTTPS request tohttps://clawhub.ai/v1/feeds/plugins.Exact steps or command run after this patch:
marketplaces.feeds.clawhub-public.url = https://clawhub.ai/v1/feeds/plugins.public-clawhub.type = clawhubandpublic-npm.type = npm.runPluginMarketplaceRefreshCommand({ feedProfile: "clawhub-public", json: true }).OPENCLAW_STATE_DIRand calledloadConfiguredHostedOfficialExternalPluginCatalogEntries(config, { feedProfile: "clawhub-public", offline: true })to verify the refresh wrote a durable snapshot.Evidence after fix:
{ "source": "hosted", "entries": 59, "metadata": { "url": "https://clawhub.ai/v1/feeds/plugins", "status": 200, "etag": "\"sha256:76a49aad745fe828c7f88b5a870ad67b5953c5235b4c928b1d6189584b257b63-gzip\"", "lastModified": "Sun, 28 Jun 2026 12:47:16 GMT", "checksum": "sha256:76a49aad745fe828c7f88b5a870ad67b5953c5235b4c928b1d6189584b257b63" }, "feed": { "id": "clawhub-official", "generatedAt": "2026-06-28T12:47:09.109Z", "sequence": 19 } }Offline snapshot verification from the same state dir:
{ "proofHead": "034ab82b95", "stateDir": "C:\\tmp\\openclaw-feeds-proof\\pr96155-refresh-034ab82", "offlineSource": "hosted-snapshot", "offlineEntries": 59, "offlineFeed": { "id": "clawhub-official", "sequence": 19 }, "offlineChecksum": "sha256:76a49aad745fe828c7f88b5a870ad67b5953c5235b4c928b1d6189584b257b63" }Observed result after fix:
The refresh command accepted the live hosted ClawHub payload, reported
source: hosted, returned 59 entries, and persisted a snapshot that the loader could read back offline from the same state directory.What was not tested:
This proof does not install plugins from the refreshed feed. Installation, search, startup consumption, feed auth, and signed-feed enforcement are later slices.