Add hosted catalog source profile validation by giodl73-repo · Pull Request #95969 · openclaw/openclaw

giodl73-repo · 2026-06-23T04:50:00Z

Summary

This PR restacks the hosted catalog source-profile slice on top of the merged persistent snapshot work in #95964.

It adds local source-profile validation for hosted catalog entries:

resolves the default clawhub-public feed profile to https://clawhub.ai/v1/feeds/plugins
keeps direct feedUrl overrides constrained to the public hosted-feed allowlist (clawhub.ai)
filters install candidates whose sourceRef is missing or unknown to local source profiles
requires manifest install source refs when a non-default or locally overridden feed profile is used
accepts trusted, available ClawHub feed entries with sourceRef: public-clawhub
normalizes ClawHub sha256:<hex> candidate integrity to SRI for the ClawHub install path without attaching that artifact hash to an npm fallback
keeps npm source-profile integrity SRI-only
fails closed for install synthesis when hosted candidates are missing explicit state: available or publisher.trust: official
validates manifest install sourceRef for custom/overridden feed profiles even when top-level candidates are present

Stack context

Merged predecessors:

Refactor external plugin catalog toward feeds #95846: feed-shaped bundled catalog fallback
Add hosted external catalog feed loader #95868: guarded hosted fetch and hosted metadata preservation
Add hosted catalog snapshot fallback #95877: in-memory last-known-good snapshot fallback contract
Persist hosted catalog snapshots in state #95964: persisted hosted snapshot cache in OpenClaw state

This PR is the next source-profile/trust validation layer. Later marketplace/config PRs must opt into any search/startup/marketplace consumption explicitly.

Boundary

Persisted hosted feed bodies remain inert cache material until this and later PRs explicitly validate source profiles and feed trust rules. This PR validates local source profiles and trusted ClawHub install candidates, but does not add broader marketplace refresh/search/startup activation.

Validation

Local validation on restacked head 6dc6ca154c7f0000a6961a347a2d572a2493b8a6:

pnpm exec oxfmt --check src/plugins/official-external-plugin-catalog.ts src/plugins/official-external-plugin-catalog.test.ts
pnpm exec oxlint --deny-warnings src/plugins/official-external-plugin-catalog.ts src/plugins/official-external-plugin-catalog.test.ts
CI=1 node scripts/test-projects.mjs src/plugins/official-external-plugin-catalog.test.ts (44 tests)
pnpm tsgo:core
git diff --check origin/main...HEAD
codex review --base origin/main on 6dc6ca154c7f0000a6961a347a2d572a2493b8a6 (no high-confidence correctness issues)

Review-driven fixes in this closeout:

8bb652b2e5: fail closed on missing state: available / publisher.trust: official and fix CI lint URL assertions.
4cbec6c7f5: avoid routing ClawHub artifact hashes into npm fallback integrity by no longer emitting npmSpec for ClawHub-source candidates.
453680c88d: validate manifest install sourceRef even when top-level candidates are present.
6dc6ca154c: keep direct feed URL overrides constrained to clawhub.ai.

Real behavior proof

Behavior or issue addressed:
Hosted ClawHub feed entries are accepted only through configured local source profiles, and trusted public-clawhub candidates normalize live ClawHub sha256:<hex> integrity values into SRI install integrity without an npm fallback sharing that artifact hash.

Real environment tested:
Windows source harness in C:\src\openclaw-prflow\.worktrees\pr-95969 at head 6dc6ca154c7f0000a6961a347a2d572a2493b8a6, using the live ClawHub feed at https://clawhub.ai/v1/feeds/plugins with no mocked feed body.

Exact steps or command run after this patch:

Ran node --import tsx - from the PR worktree.
The harness called loadHostedOfficialExternalPluginCatalogEntries({ snapshotStore: null }) and inspected resolved install metadata with resolveOfficialExternalPluginInstall(entry).

Evidence after fix:

{
  "url": "https://clawhub.ai/v1/feeds/plugins",
  "source": "hosted",
  "entryCount": 59,
  "feed": {
    "id": "clawhub-official",
    "schemaVersion": 1,
    "sequence": 17,
    "generatedAt": "2026-06-28T01:24:39.896Z"
  },
  "metadata": {
    "url": "https://clawhub.ai/v1/feeds/plugins",
    "status": 200,
    "etag": "\"sha256:ba687b54daeb2ed65e3f963e5be86214bd0421be6d19cbc76595d36374911a5d-gzip\"",
    "lastModified": "Sun, 28 Jun 2026 01:24:46 GMT",
    "checksum": "sha256:ba687b54daeb2ed65e3f963e5be86214bd0421be6d19cbc76595d36374911a5d"
  },
  "installableSample": [
    {
      "id": "@expediagroup/expedia-openclaw",
      "sourceRefs": ["public-clawhub"],
      "install": {
        "clawhubSpec": "clawhub:@expediagroup/[email protected]",
        "defaultChoice": "clawhub",
        "expectedIntegrity": "sha256-s1XdoEQDvsqri7qwaf0eewV4Ji50WeWYzFsZYVtb2rk="
      }
    }
  ],
  "unknownSourceEntryCount": 0
}

Observed result after fix:
The live hosted feed loaded as hosted, returned 59 entries, preserved the clawhub-official feed metadata, exposed trusted public-clawhub install candidates as ClawHub installs, normalized the live ClawHub checksum to SRI for the ClawHub install path without emitting an npm fallback for that artifact hash, and did not retain entries with unknown source refs.

What was not tested:
No end-user marketplace UI or startup activation path was tested; this PR is limited to source-profile validation and catalog install metadata resolution.

clawsweeper · 2026-06-23T19:16:00Z

Codex review: needs maintainer review before merge. Reviewed June 27, 2026, 11:52 PM ET / 03:52 UTC.

Summary
The PR adds hosted catalog feed/source profile defaults, sourceRef validation and filtering, and ClawHub/npm install-candidate integrity normalization in the official external plugin catalog and tests.

PR surface: Source +378, Tests +585. Total +963 across 2 files.

Reproducibility: not applicable. this is a feature and trust-boundary PR, not a bug report. The behavior is source-reviewable, covered by focused tests, and demonstrated by current-head live output in the PR body.

Review metrics: 2 noteworthy metrics.

Hosted Profile Defaults: 1 feed profile added, 2 source profiles added. These names define local trust vocabulary that dependent hosted marketplace slices can consume.
Hosted Loader Selection Inputs: 2 optional inputs added. feedProfile and catalogConfig expand caller-controlled host and source-profile selection before merge.

Stored data model
Persistent data-model change detected: database schema: src/plugins/official-external-plugin-catalog.test.ts, vector/embedding metadata: src/plugins/official-external-plugin-catalog.test.ts. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: canonical
Canonical: #95969
Summary: This PR is the canonical source-profile validation slice in the hosted marketplace feed stack; related items are prerequisites or dependent activation/configuration slices, not replacements.

Members:

adjacent_distinct: Refactor external plugin catalog toward feeds #95846 - Merged prerequisite that reshaped the bundled external plugin catalog as feed-like fallback data.
adjacent_distinct: Add hosted external catalog feed loader #95868 - Merged prerequisite that added the guarded hosted feed loader and ClawHub feed URL contract.
adjacent_distinct: Add hosted catalog snapshot fallback #95877 - Merged prerequisite that introduced hosted snapshot fallback semantics.
adjacent_distinct: Persist hosted catalog snapshots in state #95964 - Merged prerequisite that persisted hosted catalog snapshots while keeping snapshots inert.
partial_overlap: Add hosted catalog config profiles #95981 - Open dependent config-profile PR builds on this source-profile contract and currently targets this branch.
2 more in the report.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

none.

Risk before merge

[P1] The new clawhub-public, public-clawhub, public-npm names and loader inputs can become compatibility contracts for dependent marketplace config, refresh, entries, install, telemetry, and search slices.
[P1] The loader now decides which hosted remote feed candidates may become trusted install metadata, so maintainer acceptance of the source-profile trust vocabulary matters even with focused tests.
[P1] Named local feed profiles can authorize a configured HTTPS host through the SSRF guard, while direct feedUrl overrides stay public-allowlisted; that host-selection boundary should be an explicit maintainer choice.

Maintainer options:

Accept Source Profiles (recommended)
Maintainers can land this after explicitly accepting clawhub-public, public-clawhub, public-npm, and the loader inputs as the hosted-marketplace trust vocabulary.
Adjust The Contract Before Merge
If the profile names, host policy, or sourceRef vocabulary should differ, ask for that narrow adjustment before dependent marketplace slices build on it.
Pause For Stack Direction
If the trust vocabulary should be decided with the dependent config PR, pause this PR until Add hosted catalog config profiles #95981 is settled.

Next step before merge

[P2] Manual review is appropriate because the protected maintainer label and hosted-feed trust contract require maintainer acceptance, not an automated repair.

Security
Cleared: No concrete security or supply-chain defect remains in the diff; the trust-boundary impact is captured as merge risk for maintainer acceptance.

Review details

Best possible solution:

Land only after maintainers accept the source-profile and hosted-feed trust vocabulary, keeping later config and CLI activation in separate reviewed slices.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this is a feature and trust-boundary PR, not a bug report. The behavior is source-reviewable, covered by focused tests, and demonstrated by current-head live output in the PR body.

Is this the best way to solve the issue?

Yes, with maintainer acceptance: the catalog loader is the right owner for sourceRef validation before later marketplace activation, and the direct URL override remains constrained to the public ClawHub allowlist.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 830467bc9306.

Label changes

Label changes:

add proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes current-head live terminal output from a Windows source harness against the live ClawHub feed showing the new hosted source-profile and install metadata behavior.
add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes current-head live terminal output from a Windows source harness against the live ClawHub feed showing the new hosted source-profile and install metadata behavior.
remove rating: 🦪 silver shellfish: Current PR rating is rating: 🐚 platinum hermit, so this older rating label is no longer current.
remove status: 📣 needs proof: Current PR status label is status: 👀 ready for maintainer look.

Label justifications:

P2: This is normal-priority hosted marketplace plumbing with limited immediate activation but real compatibility and trust-boundary implications.
merge-risk: 🚨 compatibility: The new profile defaults and loader inputs can become contracts for later marketplace config, refresh, listing, install, telemetry, and search behavior.
merge-risk: 🚨 security-boundary: The diff changes which remote feed entries and sourceRef names may become trusted install metadata for hosted catalog entries.
rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes current-head live terminal output from a Windows source harness against the live ClawHub feed showing the new hosted source-profile and install metadata behavior.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes current-head live terminal output from a Windows source harness against the live ClawHub feed showing the new hosted source-profile and install metadata behavior.

Evidence reviewed

PR surface:

Source +378, Tests +585. Total +963 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	400	22	+378
Tests	1	591	6	+585
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	991	28	+963

What I checked:

Repository policy applied: Root policy treats plugin APIs, config/default additions, fallback behavior, and hosted trust boundaries as compatibility- and upgrade-sensitive review surfaces. (AGENTS.md:31, 830467bc9306)
Scoped plugin policy applied: The scoped plugin guide says src/plugins owns plugin discovery, manifest validation, loading, registry assembly, and contract enforcement, so this is the right code area but a contract-sensitive change. (src/plugins/AGENTS.md:1, 830467bc9306)
Current main baseline: Current main has the default ClawHub feed URL and host allowlist but no feed/source profile config or sourceRef filtering in the hosted loader. (src/plugins/official-external-plugin-catalog.ts:175, 830467bc9306)
PR head profile defaults: PR head adds the clawhub-public feed profile and public-clawhub/public-npm source profiles that later marketplace slices may consume as trust vocabulary. (src/plugins/official-external-plugin-catalog.ts:201, 6dc6ca154c7f)
PR head feed source selection: Direct feedUrl overrides remain constrained to the clawhub.ai allowlist, while named local feed profiles authorize their configured HTTPS host through the guarded fetch path. (src/plugins/official-external-plugin-catalog.ts:313, 6dc6ca154c7f)
PR head sourceRef validation: The new validation checks candidate sourceRef values against configured sources and the hosted loader filters fresh and snapshot entries through that validation before returning install metadata. (src/plugins/official-external-plugin-catalog.ts:422, 6dc6ca154c7f)

Likely related people:

giodl73-repo: The merged hosted feed, snapshot fallback, and persisted snapshot commits all came through the same catalog surface that this PR extends. (role: feature stack author and recent area contributor; confidence: high; commits: 4010b81a77f4, c29e1fe764f3, 12685ee6b752; files: src/plugins/official-external-plugin-catalog.ts, src/plugins/official-external-plugin-catalog.test.ts, src/plugins/official-external-plugin-catalog-snapshot-store.ts)
Patrick-Erichsen: Live PR metadata shows approvals on this PR and nearby hosted-feed stack PRs, making this a useful routing signal for the trust-contract review. (role: reviewer; confidence: medium; files: src/plugins/official-external-plugin-catalog.ts, src/plugins/official-external-plugin-catalog.test.ts)
joshavant: Blame on current main shows the pre-hosted catalog baseline in the grafted baseline commit, which is useful context for inherited catalog behavior. (role: adjacent baseline contributor; confidence: medium; commits: 898ca9741cba; files: src/plugins/official-external-plugin-catalog.ts, src/plugins/official-external-plugin-catalog.test.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

giodl73-repo · 2026-06-24T13:59:17Z

@clawsweeper re-review

giodl73-repo · 2026-06-28T03:03:16Z

@clawsweeper re-review

clawsweeper · 2026-06-28T03:03:18Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

giodl73-repo · 2026-06-28T03:10:32Z

@clawsweeper re-review

clawsweeper · 2026-06-28T03:10:35Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

giodl73-repo · 2026-06-28T03:22:28Z

@clawsweeper re-review

clawsweeper · 2026-06-28T03:22:31Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

giodl73-repo · 2026-06-28T03:38:23Z

@clawsweeper re-review

clawsweeper · 2026-06-28T03:38:26Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

giodl73-repo · 2026-06-28T03:46:49Z

@clawsweeper re-review

giodl73-repo · 2026-06-28T03:57:01Z

Merged via squash.

Prepared head SHA: 6dc6ca154c7f0000a6961a347a2d572a2493b8a6
Merge commit: 1b0766080a30bd976217bea0b2b87efba439734a

Thanks @giodl73-repo!

openclaw-barnacle Bot added size: M maintainer Maintainer-authored PR labels Jun 23, 2026

giodl73-repo force-pushed the feeds-hosted-catalog-source-profiles branch 2 times, most recently from e8762f2 to 623a159 Compare June 23, 2026 05:05

openclaw-barnacle Bot added size: L and removed size: M labels Jun 23, 2026

giodl73-repo force-pushed the feeds-hosted-catalog-source-profiles branch 3 times, most recently from 9464bb1 to 7ab496e Compare June 23, 2026 05:18

Patrick-Erichsen approved these changes Jun 23, 2026

View reviewed changes

giodl73-repo force-pushed the feeds-hosted-catalog-source-profiles branch 2 times, most recently from 21ef7fb to ceef5a5 Compare June 23, 2026 13:49

giodl73-repo force-pushed the feeds-hosted-catalog-persistent-snapshot branch from c371b39 to f261e76 Compare June 23, 2026 16:11

giodl73-repo force-pushed the feeds-hosted-catalog-source-profiles branch from ceef5a5 to 30fbce9 Compare June 23, 2026 16:12

This was referenced Jun 23, 2026

Add marketplace feed refresh command #96155

Merged

Add marketplace feed entries command #96158

Merged

giodl73-repo force-pushed the feeds-hosted-catalog-source-profiles branch 2 times, most recently from 3032d05 to c3348ec Compare June 24, 2026 03:51

giodl73-repo force-pushed the feeds-hosted-catalog-source-profiles branch from 8bb652b to 4cbec6c Compare June 28, 2026 03:10

giodl73-repo force-pushed the feeds-hosted-catalog-source-profiles branch from 4cbec6c to 453680c Compare June 28, 2026 03:22

clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Jun 28, 2026

fix(plugins): accept ClawHub marketplace feed v2

6dc6ca1

giodl73-repo force-pushed the feeds-hosted-catalog-source-profiles branch from 453680c to 6dc6ca1 Compare June 28, 2026 03:37

clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. labels Jun 28, 2026

github-actions Bot mentioned this pull request Jun 28, 2026

📡 Upstream Digest — 2026-06-28 07:44 UTC curtismercier/openclaw-mods#1164

Open

clawsweeper Bot mentioned this pull request Jun 28, 2026

Add marketplace feed telemetry #96194

Merged

Uh oh!

Uh oh!

Conversation

giodl73-repo commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Stack context

Boundary

Validation

Real behavior proof

Uh oh!

clawsweeper Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

giodl73-repo commented Jun 24, 2026

Uh oh!

giodl73-repo commented Jun 28, 2026

Uh oh!

clawsweeper Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

giodl73-repo commented Jun 28, 2026

Uh oh!

clawsweeper Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

giodl73-repo commented Jun 28, 2026

Uh oh!

clawsweeper Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

giodl73-repo commented Jun 28, 2026

Uh oh!

clawsweeper Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

giodl73-repo commented Jun 28, 2026

Uh oh!

giodl73-repo commented Jun 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

giodl73-repo commented Jun 23, 2026 •

edited

Loading

clawsweeper Bot commented Jun 23, 2026 •

edited

Loading

clawsweeper Bot commented Jun 28, 2026 •

edited

Loading

clawsweeper Bot commented Jun 28, 2026 •

edited

Loading

clawsweeper Bot commented Jun 28, 2026 •

edited

Loading

clawsweeper Bot commented Jun 28, 2026 •

edited

Loading