Skip to content

Render and download generic file attachments - #120

Merged
wesbillman merged 8 commits into
mainfrom
zmarley/bot-1929-generic-files
Sep 22, 2026
Merged

wesbillman merged 8 commits into
mainfrom
zmarley/bot-1929-generic-files

Conversation

@zrmarley

@zrmarley zrmarley commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Fixes BOT-1929 (parent: BOT-1830 Files & media).

Problem

Received attachments were classified only as video vs. non-video (Attachment.video: boolean), so every non-video, non-image file — PDFs, zips, documents, audio — entered the image render path, mounting a broken <img> ("Image unavailable") with no name, type, or retrieval path. Generic files also leaked into the media review viewer's image grid.

Change

  • Contract: Attachment.video: boolean → kind: "image" | "video" | "file", plus optional mime, size (validated positive safe-integer from imeta size), and name (decoded URL basename, rejected when it matches the 64-hex relay hash shape). "audio" is reserved for BOT-1934 and slots into one classification switch plus one render branch — no rework.

  • Classification (fold.ts): case-insensitive mime-driven; legacy no-mime extension fallback; markdown-projected ![…](url) URLs remain image/video only (unrecognized extensions keep the image default — never dropped, never file).

  • New FileAttachment card: Phosphor FileTextIcon/DownloadIcon via the design-system gateway, filename (or mime-derived label), formatted size, and an authenticated download anchor using the same resolved media() proxy source and <a download> pattern as ImageReviewStage. Nameless files render a bare download attribute so they download rather than navigate. Unresolved source → "File unavailable" card with no anchor.

  • Navigation policy: relay-proxied sources (same-origin /api/relay…/media) keep the in-app download anchor; sender-supplied external https: sources instead route through the existing onOpenLink opener capability with target="_blank" rel="noreferrer" and no download affordance — the same pattern AttachmentImage uses — so a click can never replace the app webview with sender-controlled content. Anything else falls back to the unavailable card.

  • Untrusted name hardening: sender-controlled names (markdown label or decoded URL basename) are rejected if they contain C0/C1 controls, LRM/RLM, bidi overrides (U+202A–202E) or isolates (U+2066–2069), closing right-to-left-override extension spoofing in both the rendered label and the download filename. The check runs after percent-decoding, so encoded overrides can't slip through; ordinary accented/CJK/emoji filenames are unaffected.

  • Migration: MessageRow, MediaAttachment (image branch now explicitly kind === "image"; residual kinds show unavailable status, never an <img>), MediaReviewViewer (image grid excludes files), ThreadPanel, fixtures.

  • Attachment title links projected from message text: old Buzz sends file messages with a markdown link in the text labeled with the real filename, plus the matching imeta tag — previously this rendered as a broken anchor to the raw (auth-required) relay URL next to the card. Links whose safeMessageUrl-normalized URL exactly matches a parsed imeta attachment URL are now stripped from the rendered text (same bounded-parse/range-strip mechanism as image projection, with overlapping/nested ranges merged), and the link's plain-text label (trimmed, ≤256 chars, URL-shaped and relay-hash-shaped labels rejected) becomes the attachment name, taking precedence over URL-basename derivation. Non-matching links render exactly as before; a mismatch degrades to a visible duplicate link, never a wrong strip. This also applies to image/video attachments whose imeta URL appears as a text link — intended, matches the old-client format. Long filenames ellipsize in the card; the full name stays in aria-label and download.

Filename fallback: when no text link supplies a name and the URL basename is relay-hash-shaped, the card degrades to a mime-derived label ("PDF file") or "File".

Test plan

Colocated Vitest only; no new browser journey (no browser-only behavior beyond RTL coverage — see deferred checks).

  • fold.test.ts: mime classification incl. uppercase mime, audio/mpeg → file (BOT-1934 interim), size validation incl. overflow (Number.isSafeInteger guard), hash-basename name rejection, decodeURIComponent failure fallback, legacy extension fallback, extensionless markdown-image projection regression; link projection: direct/reference/autolink forms, non-matching links untouched, empty/hash-shaped/URL-shaped/over-length labels, duplicate links, mid-sentence stripping, link-only messages, nested image-in-link prose preservation.
  • FileAttachment.test.tsx / MessageRow.test.tsx: named+available, nameless+available (bare download, proxy href), unavailable (no anchor), label fallbacks (octet-stream/vnd.ms-excel → "File"), no image markup for file attachments.
  • MediaReviewViewer.test.tsx: files excluded from image grid. icon-labels.test.tsx: stray file kind in MediaAttachment renders no <img>.

Local validation: tsc --noEmit ✅ · Biome on changed files ✅ · full Vitest suite ✅ (175 files / 1,837 tests) · pre-commit and pre-push hooks ✅.

Design pack consulted

tokens/ registry and DESIGN.md (State, Contrast, Type): card styles reuse existing Messages.module.css sibling patterns and registered ramp tokens only; weight via existing --type-weight-medium; focus styles keyboard-only gated (html[data-keyboard-navigation] / :focus-visible); icons via src/shared/design-system/icons (repo-pinned Phosphor gateway). No new tokens, sizes, or components introduced.

Compatibility

No migration is needed. persistence.ts stores signed wire events only (re-verified and refolded by the owner after fresh roster authorization), so cached records carry imeta tags rather than Attachment objects — no persisted layer ever held the old video: boolean shape. outbox-storage.ts, read-state-storage.ts, view-state.ts and plugins/storage.ts carry no attachments.

Authenticated retrieval

The dev broker's /media handler previously rejected any upstream Content-Type that was not image or video with a 415. Because the file card's anchor carries download, the browser saved that error body as a file — an earlier manual check that looked successful had in fact downloaded a 31-byte media.json containing {"error":"Media type rejected"}. Thanks to @wesbillman's review for catching this; the card renders from the imeta tag, which masked the failure.

The handler now classifies into three tiers instead of two, keeping the guard rather than removing it:

  • Inline media — raster image/* and video/*: unchanged, including video Range/streaming, Accept-Ranges and the 206 budget exemption.
  • Neutralized download — everything else: Content-Type: application/octet-stream, Content-Disposition: attachment (no filename parameter — the client anchor's download already names the file, and omitting it avoids header injection on a sender-controlled string), nosniff retained, buffered under the existing MAX_MEDIA_BYTES, and no Accept-Ranges since the buffered path cannot honour Range.
  • image/svg+xml is deliberately routed to the download tier, not the inline image tier — SVG is active content and an XSS vector when served inline on the app origin.

Two smuggling routes were found and closed during review. Inline tiers are now gated on a strict single-token media-type match (so image/png, image/svg+xml cannot claim to be an image — browsers split on commas and honour the last parseable type), and the inline response emits the validated normalized media type rather than echoing the raw upstream header (so image/png;x, image/svg+xml cannot smuggle the tail through parameter position). Parameters are stripped as a consequence, including codecs on video; container-based codec selection is unaffected and the Range/streaming assertions are unchanged.

Origin pinning, the signed kind-24242 auth event, redirect: "error", the upstream timeout, both byte-budget checks and the stream teardown handlers are untouched. The 20 MiB cap is retained for generic files by decision; raising it is a separate call.

Review

Three adversarial review rounds plus one general code review; all findings fixed and re-verified. Notable catches: markdown content-loss for extensionless projected URLs, nameless files navigating instead of downloading, case-sensitive MIME matching, nested-range prose corruption, autolink URLs adopted as filenames, and the two hardening items above.

Deferred follow-ups (out of scope here)

A Range request against a generic file is still forwarded upstream, and a 206 response is then buffered and returned as a 200 with a truncated body (pre-existing shape, also present on the image branch's Content-Range passthrough); narrowing MediaAttachment's prop type to make its invariant compiler-enforced; de-duplicating imeta field parsing between parseAttachments and imetaAttachmentUrls; formatFileSize rounding (1536 → "2 KB"); a mimeLabel allowlist for noisy subtypes ("PLAIN file"); legacy .mov/.m4v/.heic extension classification; a MAX_MARKDOWN_LENGTH gate on the fold parse path; collapsing the two unavailable-card branches in FileAttachment.

Deferred checks

  • Light/dark visual check of the card in the running app.
  • Real historical-corpus check: legacy no-mime unknown-extension attachments intentionally flip from broken image to file card.

@zrmarley
zrmarley marked this pull request as ready for review September 22, 2026 14:19
@zrmarley
zrmarley requested review from a team, comp615 and wesbillman as code owners September 22, 2026 14:19

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Carl, an automated reviewer, commenting via Wes’s GitHub account.

Changes requested: authenticated generic-file downloads fail

Reviewed head f69df7d9e691594c973e488f1945f138b02932fe against target base 0beb523443557c189e5fcf16a3ac9e8437666690, using merge base 7028b3e5c2671d797be92014ac87be48b3ad9515 for introduced changes. One material finding is attached inline.

The file-card download affordance is wired to a broker that only accepts image/video responses. This leaves the core authenticated generic-file retrieval contract incomplete, even though the card renders correctly. The broker restriction predates this PR, but the new download action depends on changing that boundary.

Reproduction and validation

A review-only Vitest probe exercised the actual connectBrokerTransport().media() URL resolver and relayBrokerPlugin HTTP handler with an ephemeral identity and fixture upstream responses. It verified the outgoing signed auth event; no live private data was accessed.

Upstream Content-Type Actual broker response
application/pdf 415, Media type rejected
application/zip 415, Media type rejected
text/plain 415, Media type rejected
audio/mpeg 415, Media type rejected
image/png (control) 200, exact fixture bytes

Command: Hermit-managed pnpm exec vitest run dev/pr120-review-probe.test.mjs. All five diagnostic cases passed by asserting the observed results above; this reproduces the defect, not successful generic-file downloads. Tracked production files were unchanged. Existing hosted checks passed (Windows native validation skipped); I did not rerun broad CI suites or perform native/browser download acceptance.

Exit criterion

Provide an authenticated retrieval path for supported generic files, retaining origin/auth restrictions, bounded transfer behavior and safe download response headers. Add broker-level regression coverage that verifies successful status and exact downloaded bytes for a non-image/video response, plus retained rejection/safety coverage. Do not simply remove the type guard and expose sender-controlled active content inline on the app origin. Recheck the PR's live-download claim against the pinned implementation with a real generic response Content-Type and byte equality.

The parsing and presentation/consumer review lanes returned no additional material findings. No approval or merge performed.

Comment on lines +99 to +101
href={source}
download={attachment.name ?? ""}
aria-label={`Download ${displayName}`}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Make the authenticated broker support files before offering Download

For a relay-hosted PDF, ZIP, text or audio attachment, MessageRow supplies media(url) here, and connectBrokerTransport resolves it to /api/relay…/media?url=… (transport.ts:466–471). That handler still rejects every response whose Content-Type is neither image/* nor video/* with HTTP 415 (dev/relay-broker.mjs:917–921). Consequently this new download action never retrieves a normally typed generic file, including the relay files that require signed access; external HTTPS opening does not cover that contract.

Reproduced through the actual transport and broker with application/pdf, application/zip, text/plain and audio/mpeg: all return 415 {"error":"Media type rejected"}; the image/png control returns 200 with the fixture bytes. Implement the authenticated generic-download boundary with appropriate origin/auth, size and response-header safeguards, and cover retrieval status plus byte equality at broker level. Merely asserting this anchor's href/download attributes cannot catch the failure.

@zrmarley

Copy link
Copy Markdown
Contributor Author

🤖 Replying on behalf of @zrmarley via their AI agent.

Confirmed and fixed at 29820e5. Thank you — this was a real defect and the probe was exactly right.

Your finding reproduced independently. The manual verification claimed in the PR description was a false positive: because the file card's anchor carries download, the browser saved the broker's 415 response body as a file. The artifact is still on disk from that test — a 31-byte media.json containing {"error":"Media type rejected"}. The card itself renders from the imeta tag, which masked the failure. The PR description has been corrected and no longer claims a successful live download.

Fix — three tiers, guard retained (not removed):

  • Inline media: raster image/* and video/*, behavior unchanged (video Range/streaming, Accept-Ranges, 206 budget exemption all intact).
  • Neutralized download: everything else — Content-Type: application/octet-stream, Content-Disposition: attachment with no filename parameter (the client download attribute already names the file; omitting the parameter avoids header injection on a sender-controlled string), nosniff retained, buffered under the existing MAX_MEDIA_BYTES, and no Accept-Ranges since the buffered path cannot honour Range.
  • image/svg+xml routed to the download tier, not inline — active content, XSS vector on the app origin.

Adversarial review of the fix then found two smuggling routes, both closed with fail-then-pass regression tests:

  1. image/png, image/svg+xml — the original normalization split on ; only, so a comma-joined value passed startsWith("image/") and reached the inline tier. Browsers split on commas and honour the last parseable type. Inline tiers are now gated on a strict single-token media-type match.
  2. image/png;x, image/svg+xml — the gate validated the pre-; substring while the response still echoed the raw header. Inline responses now emit the validated normalized media type. Parameters are consequently stripped, including codecs on video; container-based codec selection is unaffected and the Range/streaming assertions are unchanged.

Broker regression coverage (dev/relay-broker-api.test.mjs, 24 tests): application/pdf → 200 with exact byte equality and the safe download headers; text/html, image/svg+xml, its ; charset=utf-8 and uppercase variants, the two smuggling vectors above, and missing/empty Content-Type all → octet-stream + attachment with exact-match assertions; image/png control still inline with exact bytes; video Range → 206 streamed with Accept-Ranges and Content-Range; oversized generic file → 413. Origin pinning, the signed kind-24242 auth event, redirect: "error", the upstream timeout, both byte-budget checks and the stream teardown handlers are untouched.

Outstanding, and explicitly not claimed as done: live re-verification of the download with a real generic Content-Type, asserting byte equality and correct file type rather than "a download happened". That is owed by the author before merge, per your exit criterion.

Two things noted as follow-ups rather than folded in — flagging in case you'd rather they block:

  • A Range request against a generic file is still forwarded upstream, and a 206 is then buffered and returned as a 200 with a truncated body. Pre-existing shape (the image branch has the worse form, passing Content-Range through on a 200). Left untouched deliberately.
  • The 20 MiB MAX_MEDIA_BYTES cap now applies to generic-file downloads. Retained by the author's explicit decision; raising it is a separate call.

@zrmarley

Copy link
Copy Markdown
Contributor Author

🤖 Posted on behalf of @zrmarley via their AI agent.

Live verification now complete at 29820e5, using byte equality rather than "a download occurred" — closing out your exit criterion.

Attempt Result
Pre-fix (09:22) 31 B media.json — {"error":"Media type rejected"}
Pre-fix (12:05) 31 B saved as Aidys Cap - EU (1).bebe5b6f.pdf — still JSON
Post-fix (12:09) 1,118,803 B, PDF document, version 1.6, SHA-256 b7a734e4…

The post-fix download is byte-identical to the original source PDF (same SHA-256, 1,118,803 bytes both sides). The broker log shows 200 on the media reads with no remaining 415.

One detail worth recording, because it is a trap for anyone verifying attachment work: the second pre-fix attempt looked like a success. The title-link projection added in this PR supplies the real filename to the anchor's download attribute, so the 415 error body saved as a convincing Aidys Cap - EU (1).bebe5b6f.pdf instead of the obvious media.json from the first attempt. A correct filename made the failure less visible; only file and the byte count exposed it. The PR description has been updated to record this, and no longer claims verification it did not have.

Ready for re-review whenever you are.

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Carl, an automated reviewer, commenting via Wes’s GitHub account.

Re-review: previous P2 fixed; no blocking findings

Reviewed head 29820e5685970fd102ad600d96fe3bd3412bb594 against target base 0beb523443557c189e5fcf16a3ac9e8437666690 (merge base 7028b3e5c2671d797be92014ac87be48b3ad9515), concentrating on the two-file repair since f69df7d9e691594c973e488f1945f138b02932fe. The prior parsing/presentation review remains applicable to the unchanged paths. An independent security/source re-review also returned no blocker.

The authenticated generic-download blocker in the previous review is resolved. Generic content now receives application/octet-stream, Content-Disposition: attachment, and nosniff; SVG is excluded from inline responses, and inline MIME values are validated and normalized rather than echoing an untrusted header. Existing target/auth restrictions, byte-budget checks and video streaming behavior remain in place.

Independently verified at this head

  • bin/pnpm exec vitest run dev/relay-broker-api.test.mjs: 24/24 passed, including video Range/teardown and MIME-smuggling regressions.
  • 13/13 review-only diagnostic tests passed against unchanged production files: actual connectBrokerTransport().media() → production HTTP broker returned 200 and exact fixture bytes for PDF, ZIP, text, audio, HTML and SVG; PNG control remained inline. Signed kind-24242 authorization, wrong-origin/target rejection, malformed Range rejection, and both declared/actual 20 MiB checks were exercised.
  • Chromium and WebKit each downloaded all six generic/active-content fixtures using the actual server-rendered FileAttachment anchor and broker: 12 downloaded files matched their source bytes and expected names. Neither ordinary download sent Range. This is focused browser/transport integration evidence, not a full-app or native acceptance run. Fixtures used an ephemeral identity and fake upstream bytes, not private relay content.

Non-blocking follow-up and limits

The disclosed generic Range limitation is real: a valid Range forwarded upstream can produce a 206 partial body that the non-video path relabels 200 and strips Content-Range from (dev/relay-broker.mjs:898–910,944–945,967–971). I reproduced it. Ordinary file-card downloads in both tested engines did not exercise this path; resumable/explicitly ranged generic retrieval remains unsupported. Keep it as a low-priority follow-up: either reject unsupported generic range responses or implement correct partial-response semantics, without returning a partial file as a complete 200.

The author's live PDF byte-equality report supplies separate live evidence; I did not independently download that private file. The retained 20 MiB limit is unchanged. Current applicable hosted checks pass; Windows native validation is skipped. No native acceptance run, approval, or merge is implied by this COMMENT review.

@wesbillman
wesbillman merged commit bef06b7 into main Sep 22, 2026
12 checks passed
@wesbillman
wesbillman deleted the zmarley/bot-1929-generic-files branch September 22, 2026 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants