Hosted Marketplace Feed Stack

This is PR3 in the hosted marketplace feed stack. The stack should land in order, with maintainer review at each step before the later PRs wire hosted marketplace data into broader user flows.

1. Refactor external plugin catalog toward feeds #95846: refactor the bundled external plugin catalog toward the feed-shaped data model.
2. Add hosted external catalog feed loader #95868: add the lazy hosted marketplace feed loader for the configured public feed URL, with guarded HTTPS fetch, checksum, schema, size, timeout, and bundled fallback behavior.
3. Add hosted catalog snapshot fallback #95877: add accepted hosted snapshot fallback so a failed or unchanged hosted fetch can reuse the last valid feed payload.
4. Persist hosted catalog snapshots in state #95964: persist hosted marketplace feed snapshots in OpenClaw state.
5. Add hosted catalog source profile validation #95969: add marketplace source profile validation for hosted feed install candidates.
6. Add hosted catalog config profiles #95981: add configurable marketplaces.feeds and marketplaces.sources profiles.
7. Add marketplace feed refresh command #96155: add openclaw plugins marketplace refresh as the explicit manual snapshot refresh path.
8. Add marketplace feed entries command #96158: add openclaw plugins marketplace entries as the explicit manual read path over hosted, snapshot, or bundled fallback entries.
9. Add marketplace feed telemetry #96194: add marketplace feed telemetry.

The broader follow-up after this stack is to decide whether and how marketplace entries should feed install/search/startup behavior, add automatic conditional refresh using ETag / Last-Modified, or move to ClawHub producer-side validation for /v1/feeds/plugins and /v1/feeds/skills.

Journey Context

This is step 3 of the hosted OpenClaw feed path from RFC 19. #95846 made the bundled catalog feed-shaped. #95868 added guarded hosted fetch. This PR adds the last-known-good snapshot contract that makes the hosted path safe when the register endpoint is unchanged or temporarily unhealthy.

Current head 5f71635bba is restacked on main after #95868 merged.

What This PR Does

• Adds a narrow HostedOfficialExternalPluginCatalogSnapshotStore interface.
• Adds an in-memory snapshot store for tests and future runtime binding.
• Writes the exact hosted response body plus hosted metadata and savedAt after a successful hosted load.
• Uses a cached snapshot on 304, non-OK hosted responses, invalid hosted JSON/schema, or fetch failures.
• Validates cached snapshot body checksum before use.
• Validates cached snapshot schema before use.
• Preserves expectedSha256 pinning for snapshots, so a stale cached body cannot satisfy a caller-pinned checksum it does not match.
• Honors request validators when serving snapshots, so conditional callers do not reuse a stale URL-only snapshot after ETag / Last-Modified changes.
• Falls back to the bundled catalog when no valid snapshot exists.

What This PR Intentionally Does Not Do

This PR does not choose a production persistence location, wire SQLite/runtime storage, enable the hosted loader by default, add source profiles, add auth, add signed envelopes, add regional feeds, or change search/install behavior. It defines the loader-level snapshot semantics that a later runtime-owned persistent store can bind to.

How To Review

Review this as fallback semantics, not as the final storage implementation. The key behavior is: hosted feed first, validated last-known-good snapshot second, bundled catalog last. A snapshot write failure is intentionally non-fatal because a good hosted response should still be usable even if cache persistence fails.

Validation

• pnpm exec oxlint --deny-warnings src/plugins/official-external-plugin-catalog.ts src/plugins/official-external-plugin-catalog.test.ts
• pnpm exec oxfmt --check src/plugins/official-external-plugin-catalog.ts src/plugins/official-external-plugin-catalog.test.ts
• git diff --check
• CI=1 node scripts/test-projects.mjs src/plugins/official-external-plugin-catalog.test.ts (33 tests)
• codex review --base origin/main reported no actionable correctness issues.
• Live guarded snapshot proof: node --import tsx - source harness against https://clawhub.ai/v1/feeds/plugins with a file-backed snapshot store under C:\tmp\openclaw-feeds-proof\pr95877-live-snapshot.

Real Behavior Proof

Behavior or issue addressed:
The hosted loader reuses a last-known-good hosted snapshot when a later hosted fetch cannot provide an acceptable fresh payload, while refusing stale snapshots when the caller supplied request validators that no longer match the snapshot metadata.

Real environment tested:
Windows checkout C:\src\openclaw-prflow\.worktrees\pr-95877 at head 5f71635bba, live guarded HTTPS fetch to https://clawhub.ai/v1/feeds/plugins, and a file-backed HostedOfficialExternalPluginCatalogSnapshotStore writing the exact snapshot body and metadata to C:\tmp\openclaw-feeds-proof\pr95877-live-snapshot\official-external-plugin-catalog.snapshot.json.

Exact steps or command run after this patch:

1. pnpm exec oxlint --deny-warnings src/plugins/official-external-plugin-catalog.ts src/plugins/official-external-plugin-catalog.test.ts
2. pnpm exec oxfmt --check src/plugins/official-external-plugin-catalog.ts src/plugins/official-external-plugin-catalog.test.ts
3. git diff --check
4. CI=1 node scripts/test-projects.mjs src/plugins/official-external-plugin-catalog.test.ts
5. codex review --base origin/main
6. node --import tsx - with a source harness that:
   • calls loadHostedOfficialExternalPluginCatalogEntries({ snapshotStore }) against the live default feed,
   • persists the successful hosted response through a file-backed snapshot store,
   • calls loadHostedOfficialExternalPluginCatalogEntries({ snapshotStore, ifNoneMatch: first.metadata.etag }), and
   • verifies the second conditional request returns source: "hosted-snapshot" with error: "hosted catalog feed returned HTTP 304".

Evidence after fix:

Test Files  1 passed (1)
Tests  33 passed (33)
[test] passed 1 Vitest shard in 27.60s

Codex review: No actionable correctness issues were found in the changed catalog snapshot logic or its adjacent tests.

Live guarded snapshot proof output:
{
  "repoHead": "5f71635bbafe75c8109248da66b297b8ee4f9657",
  "feedUrl": "https://clawhub.ai/v1/feeds/plugins",
  "snapshotPath": "C:\\tmp\\openclaw-feeds-proof\\pr95877-live-snapshot\\official-external-plugin-catalog.snapshot.json",
  "firstLoad": {
    "source": "hosted",
    "status": 200,
    "entries": 59,
    "feedId": "clawhub-official",
    "sequence": 16,
    "etag": "\"sha256:cf3d85c1d6ede1907ca265e9b214ac6ff1c715c90c30ec65e8017addb590e32d-gzip\"",
    "lastModified": "Sat, 27 Jun 2026 18:43:36 GMT",
    "checksum": "sha256:cf3d85c1d6ede1907ca265e9b214ac6ff1c715c90c30ec65e8017addb590e32d"
  },
  "secondConditionalLoad": {
    "requestIfNoneMatch": "\"sha256:cf3d85c1d6ede1907ca265e9b214ac6ff1c715c90c30ec65e8017addb590e32d-gzip\"",
    "source": "hosted-snapshot",
    "status": 200,
    "entries": 59,
    "feedId": "clawhub-official",
    "sequence": 16,
    "error": "hosted catalog feed returned HTTP 304",
    "snapshotSavedAt": "2026-06-27T20:54:47.927Z"
  },
  "persistedSnapshot": {
    "url": "https://clawhub.ai/v1/feeds/plugins",
    "savedAt": "2026-06-27T20:54:47.927Z",
    "etag": "\"sha256:cf3d85c1d6ede1907ca265e9b214ac6ff1c715c90c30ec65e8017addb590e32d-gzip\"",
    "checksum": "sha256:cf3d85c1d6ede1907ca265e9b214ac6ff1c715c90c30ec65e8017addb590e32d",
    "bytes": 21942
  }
}

Observed result after fix:
The live proof first loaded the ClawHub feed over the guarded hosted path and persisted a snapshot. The second load sent the returned live ETag as If-None-Match, received HTTP 304 from the live service, validated the persisted snapshot metadata/body, and returned source: "hosted-snapshot" with the same 59-entry clawhub-official sequence 16 feed.

What was not tested:
This PR still does not test production SQLite persistence because that is the next PR in the stack. This proof uses a file-backed implementation of the snapshot-store interface to exercise the loader contract with the live hosted feed.