fix: allow trusted host-read html after outbound staging#90579
Conversation
|
Codex review: needs maintainer review before merge. Reviewed July 12, 2026, 6:45 AM ET / 10:45 UTC. Summary PR surface: Source +195, Tests +394, Generated +19. Total +608 across 8 files. Reproducibility: yes. Current source traces trusted temporary-root HTML through outbound staging to a second host-read rejection, and independent macOS and Linux production reports reproduce the same behavior. Review metrics: 1 noteworthy metric.
Stored data model Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Risk before merge
Maintainer options:
Next step before merge
Maintainer decision needed
Security Review detailsBest possible solution: Merge the focused content-bound implementation after the trust-boundary owner explicitly accepts persisted exact-byte provenance; track row-bounding and broader send-stage graceful degradation separately if desired. Do we have a high-confidence way to reproduce the issue? Yes. Current source traces trusted temporary-root HTML through outbound staging to a second host-read rejection, and independent macOS and Linux production reports reproduce the same behavior. Is this the best way to solve the issue? Yes technically. Content-bound read-once provenance is the narrowest maintainable repair found and is substantially safer than trusting the outbound directory by path; only security-policy acceptance remains. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 75f0960b71be. Label changesLabel justifications:
Evidence reviewedPR surface: Source +195, Tests +394, Generated +19. Total +608 across 8 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
Review history (3 earlier review cycles) |
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
Patch ready for maintainer reviewThe patch has converged through three iterations addressing each ClawSweeper round:
The current head:
Two remaining gates need maintainer judgement1. Security boundary acceptance. ClawSweeper P1: "Merging expands the trusted host-read HTML boundary from temp-root files to outbound-staged files with a SQLite provenance marker, so maintainers need to accept that security lifecycle explicitly." The expansion is provenance-gated (only paths that the host-read gate already trusted at source time are marked), and the marker is best-effort cleared by the same cleanup pass that removes the file. Asking explicitly: is this trust lifecycle acceptable, or is there a tighter binding you'd prefer? 2. Real behavior proof. ClawSweeper has been asking for a Mantis Telegram Desktop capture across all three iterations; only a maintainer can trigger it. If you'd like to fire it, the suggested invocation (per ClawSweeper's own guidance) is: @openclaw-mantis telegram desktop proof: verify a generated MEDIA:/tmp/openclaw/*.html report is delivered as a Telegram document after outbound staging. @steipete — pinging since you authored the original trusted-generated-HTML host-read seam (commit 68d0c0f) and have the strongest context for the trust-boundary acceptance call. |
Targeted follow-up after the latest ClawSweeper verdictClawSweeper has now re-reviewed the current head (
The remaining gate is proof / maintainer acceptance of the intentional trust-boundary expansion:
Why the previous Mantis proof did not exercise this bugThe previous Mantis artifact showed both baseline and candidate delivering HTML. That is consistent with the earlier diagnosis: the Mantis SUT did not enable the host-read gate for the second staged reload path, so both lanes bypassed For this regression, the proof stimulus must explicitly hit this sequence:
Requested targeted proof@openclaw-mantis telegram desktop proof: please verify PR #90579 with host-read enabled on the staged reload path. The stimulus should drive a generated Useful acceptance signals for the artifact/logs:
Maintainer/security acceptance askIf the targeted proof is sufficient, could a maintainer or security owner explicitly accept this lifecycle? The proposed boundary is intentionally narrow: temp-root generated HTML remains unchanged; outbound-staged HTML is trusted only when (a) it was staged from a previously trusted generated-HTML source, (b) the staged realpath has a SQLite provenance row, and (c) the bytes read during host-read match the recorded |
2dcfc94 to
4f4dccc
Compare
- Remove dead `clearTrustedGeneratedHtmlMarker` export (lifecycle already owned by `pruneStaleTrustedGeneratedHtmlMarkers` via `cleanOldMedia`). - `markTrustedGeneratedHtmlPath` now asserts the realpath is inside the outbound staging dir; tightens the API contract independently of the reader-side containment check. - Outbound attachment staging tolerates ENOENT race on marker write (file unlinked between save and mark): warn + continue; downstream gate still rejects if the file actually vanished. - `hasTrustedGeneratedHtmlMarker` catches state-DB failures (locked, corrupt, permissions) and returns false, so a broken DB degrades to the existing path-not-allowed gate rejection rather than a thrown non-LocalMediaAccessError. - Add `logVerbose` for trust-gate decisions and prune sweep counts; the original bug surfaced as a silent drop, observability would have shortened the diagnostic loop. - Add regression test for symlink-escape: marker call throws and the reader-side gate rejects even with a forged provenance row. Addresses post-review concerns from independent review at openclaw#90579 .
|
@clawsweeper re-review Rebased onto current |
|
🦞🧹 I asked ClawSweeper to review this item again. |
4f4dccc to
96f7e87
Compare
- Remove dead `clearTrustedGeneratedHtmlMarker` export (lifecycle already owned by `pruneStaleTrustedGeneratedHtmlMarkers` via `cleanOldMedia`). - `markTrustedGeneratedHtmlPath` now asserts the realpath is inside the outbound staging dir; tightens the API contract independently of the reader-side containment check. - Outbound attachment staging tolerates ENOENT race on marker write (file unlinked between save and mark): warn + continue; downstream gate still rejects if the file actually vanished. - `hasTrustedGeneratedHtmlMarker` catches state-DB failures (locked, corrupt, permissions) and returns false, so a broken DB degrades to the existing path-not-allowed gate rejection rather than a thrown non-LocalMediaAccessError. - Add `logVerbose` for trust-gate decisions and prune sweep counts; the original bug surfaced as a silent drop, observability would have shortened the diagnostic loop. - Add regression test for symlink-escape: marker call throws and the reader-side gate rejects even with a forged provenance row. Addresses post-review concerns from independent review at openclaw#90579 .
|
Hi @vincentkoc — sorry for the direct tag; I'm trying to route this to the right maintainer rather than bump the PR blindly. This PR only touches the media outbound/trust-gate path, and I noticed you've recently worked on |
Independent production repro of #90557 + security review of this fixContext: hit this bug in production today and root-caused it independently before finding #90557 — our conclusion matches the issue exactly. Sharing the repro as a second environment data point, plus a review of the fix design. (Reviewed at commit 96f7e87.) Independent reproduction (Linux, v2026.6.8, Telegram direct chat)Environment: OpenClaw 2026.6.8 (844f405), Node v22.22.1, Linux 6.17 (GCP), Telegram bot account, direct chat, single-agent setup.
Review of the fix design (content-bound provenance)Read the full diff. The core design — bind trust to
Three minor points, none blocking:
CI is green on the PR head (check-test-types / check-lint / build-artifacts / QA Smoke), and the added tests cover replacement, forged-row, symlink, prune-lockstep, and marker-write-failure paths. We could not rebuild the PR locally (disk constraints on the repro box), so this review relies on CI for the unit suite and on the dist-level harness above for the behavioral before-state. Verdict: the design is correct and fail-closed; would like to see this merged. The production data above hopefully helps the security review along — this bug currently causes silent, unrecoverable loss of entire replies on a default-ish Linux + Telegram setup. Disclosure: AI-assisted — the repro harness, log analysis, and review draft were produced with an AI agent under my direction; I reviewed the findings and evidence before posting. Flagging per the repo's AI-transparency guideline. Update (same day): empirical confirmation that the second gate is the only blocker. To unblock our own instance we deployed a minimal local variant of this fix onto the shipped 2026.6.8 dist — path-trust for the outbound staging dir only (no provenance table; deliberately weaker than this PR, single-user box). Result, same Linux + Telegram setup as the repro above:
This is not a test of this PR's code — but it empirically confirms the PR's core premise: once the staged copy is recognized as trusted, delivery works end-to-end; there is no additional hidden gate downstream of the one this PR fixes. We'll drop our local variant in favor of this PR's content-bound design as soon as it ships. (Same AI-assisted disclosure as above applies.) |
|
Follow-up: independent local verification of this PR (built at 96f7e87) Earlier I posted an independent repro of #90557 (comment). I've now built this branch locally and verified the fix at four levels: 1. PR unit tests — 2. 3. Independent no-mock verification — my own test (real fs + real SQLite + real
4. Real-send E2E — using this PR build against the live Telegram Bot API (isolated One tip for other verifiers: don't start a second gateway against the same bot token to test this — it competes for From my side this fixes the production silent-drop we hit, and the worst-case path is now loud instead of silent. LGTM. |
Rebased onto current
|
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
Rebased onto current
|
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
|
This assigned pull request has been automatically marked as stale after being open for 27 days. |
Co-authored-by: wanglu241 <[email protected]>
|
ClawSweeper status: review started. I am starting a fresh review of this pull request: fix: allow trusted host-read html after outbound staging This is item 1/1 in the current shard. Shard 0/1. This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking. Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted. |
|
Merged via squash.
|
What Problem This Solves
Fixes #90557. Trusted generated HTML reports pass the initial host-read check under the OpenClaw temp root, but normal reply delivery copies them into
media/outboundand a channel adapter later reloads that staged path with host-read enabled. The staged path is outside the trusted temp root, so the second load rejects it and the outbound reply can be lost.The trust decision must survive a process boundary. Normal reply delivery can persist a write-ahead queue intent before platform I/O, and gateway startup reloads pending delivery payloads for retry. An in-memory marker would therefore be insufficient for restart recovery.
Why This Change Was Made
When
resolveOutboundAttachmentFromUrlstages HTML that already passed the generated-HTML trust gate, it records the staged file's canonical realpath, SHA-256, byte size, provenance kind, version, and creation time in the shared SQLite state DB.loadWebMediaaccepts outbound HTML only when containment, the provenance row, and the exact buffer already read all agree.This does not trust the outbound directory by path. The existing regular-file, no-symlink, single-link, text-shape, and HTML-document checks remain in force. Outbound precedence also prevents an overlapping configured temp/state root from bypassing provenance. Marker-write failure removes the staged file and propagates; media cleanup prunes stale rows in bounded batches.
User Impact
Generated
MEDIA:/tmp/openclaw/*.htmlreports can be delivered after outbound staging instead of being rejected by the channel adapter's second host-read check. Arbitrary, unmarked, or byte-replaced HTML undermedia/outboundremains rejected.Evidence
Exact head:
28fe171dd9666352e19c59443e32cf8896734be3, rebased ond823cdc79496877883d8bba75691eac2e633fb99.Regression coverage includes exact-byte acceptance, same-size replacement rejection, unmarked outbound rejection, overlapping-root precedence, marker-write cleanup, stale-row pruning, transient filesystem-error retention, and refusal to mark paths outside outbound staging.
The original report reproduces the first trusted load followed by the rejected staged reload. Independent Linux reproduction and a real Telegram Bot API send in the PR discussion confirm the same gate and end-to-end delivery behavior.
Contributor credit is preserved in the commit as
Co-authored-by: wanglu241 <[email protected]>.