Skip to content

fix(media): keep transparency in image thumbnails with WebP - #9003

Merged
johnmatthewtennant merged 2 commits into
mainfrom
jtennant/simone/webp-thumbnails
Nov 4, 2026
Merged

johnmatthewtennant merged 2 commits into
mainfrom
jtennant/simone/webp-thumbnails

Conversation

@johnmatthewtennant

@johnmatthewtennant johnmatthewtennant commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This fixes transparent images showing a black background. Image thumbnails were always JPEG, which can't store transparency, so a transparent PNG or GIF upload, such as an agent avatar, rendered on black wherever a client showed its thumbnail.

The relay now writes a lossy WebP thumbnail at {sha256}.thumb.webp, which keeps transparency, alongside the existing {sha256}.thumb.jpg:

  • Encoding: the WebP comes from libwebp through the webp crate, built from vendored C source, so the relay needs no system library. A GIF uses its first frame.
  • Upload responses: the upload descriptor's thumb, and its per-tenant rewrite, point to the WebP.
  • JPEG kept: installed clients derive .thumb.jpg themselves, so the relay keeps writing it until clients move to WebP with a JPEG fallback. Existing images still have only their JPEG thumbnail.
  • Serving: /media/ serves both variants, each with its own content type. Other .thumb.* extensions still return 404.
  • imeta: a thumb may name either variant, and the relay checks that the named object exists.
  • Storage sweep: the bucket taxonomy counts both variants as the image's thumbnail bytes instead of classifying .thumb.webp as unknown.
320px thumbnail JPEG PNG lossy WebP (q80)
Transparent avatar (starter-team fizz.png) 11.3 KB, black background 65.1 KB 13.8 KB
Opaque avatar (sprout-agent.png) 23.6 KB 156.5 KB 12.3 KB
Screenshot (channel-thread.png) 13.4 KB 62.5 KB 6.4 KB

Encoding a WebP thumbnail takes 3 to 20 ms.

Related issue

Supersedes #8772, which served a PNG at the .thumb.jpg path.

Testing

Red/green against a local relay on :3000, built from the branch under test, with Postgres, Redis and MinIO from docker compose:

cargo test -p buzz-test-client --test e2e_media test_transparent_png_thumbnail_is_webp_with_alpha -- --ignored
cargo test -p buzz-test-client --test e2e_media_extended test_ws_imeta_webp_thumb -- --ignored
  • On main: both tests fail. The descriptor's thumb is .thumb.jpg, and the WebSocket send is rejected with invalid: imeta thumb must be a local .thumb.jpg path.
  • On this branch: both tests pass:
    • Uploading a half-transparent PNG returns a thumb ending .thumb.webp.
    • HEAD and GET on that URL return Content-Type: image/webp.
    • The decoded thumbnail keeps alpha 0 on the transparent half and alpha 255 on the opaque half.
    • .thumb.jpg still returns image/jpeg.
    • A kind 9 message whose imeta thumb names the .thumb.webp is accepted.

Screenshots

The starter-team fizz.png avatar uploaded to a relay built from this branch. Both thumbnails are shown as round avatars beside the original PNG. .thumb.jpg fills the transparent area with a solid disc, and .thumb.webp keeps the transparency.

Transparent avatar thumbnails: .thumb.jpg, .thumb.webp and the original PNG on light and dark backgrounds

🤖 Generated with Claude Code

Image thumbnails were always JPEG, so transparent PNG and GIF uploads,
such as agent avatars, rendered on black.

The relay now writes a lossy WebP thumbnail through libwebp (the `webp`
crate, built from vendored source) at `{sha256}.thumb.webp`, and points
the upload descriptor and its tenant rewrite at it. It still writes
`{sha256}.thumb.jpg` for installed clients that derive that path.
`/media/` serves both variants with their own content types, imeta
accepts either as `thumb` and checks the named object exists, and the
bucket taxonomy counts both as thumbnails.

Refs AIDA-2308

Co-authored-by: Simone <0828be588d5c9c4d092c5b1cf01761d5aef63d6b104807fd8e8284d517dd1a99@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: John Matthew Tennant <[email protected]>
@johnmatthewtennant
johnmatthewtennant marked this pull request as ready for review November 3, 2026 17:31
@johnmatthewtennant
johnmatthewtennant requested a review from a team as a code owner November 3, 2026 17:31
@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Nov 3, 2026
@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

🔐 Codex Security Review

Note: This is an automated, security-focused review generated by Codex.
Use it as a supplement to human review; false positives are possible.

Scope

  • Exact PR diff: 0e5a69962e5cbe5e81ea46b557770ce27a120a14...05561b406935c6b976208e9bb9e08a9396418824
  • Model: gpt-6.1-sol

💡 Click "edited" above to see earlier reviews for this PR.


Review Summary

Overall Risk: NONE

No concrete security, correctness, or reliability regressions found in the authorized PR range.

Findings

No concrete security, correctness, or reliability findings were identified.

Notes

  • Review used read-only source and history inspection. Builds and tests were not executed, as instructed.

Generated by Codex Security Review |
Requested by: @johnmatthewtennant |
Workflow run

@johnmatthewtennant

Copy link
Copy Markdown
Contributor Author

Screenshots

Transparent avatar thumbnails

The starter-team fizz.png avatar uploaded to a relay built from this branch, with both thumbnails it serves shown as round avatars beside the original PNG. .thumb.jpg fills the transparent area with a solid disc, and .thumb.webp keeps the transparency.

transparent-avatar-thumbnails

johnmatthewtennant added a commit that referenced this pull request Nov 3, 2026
Co-authored-by: Simone <0828be588d5c9c4d092c5b1cf01761d5aef63d6b104807fd8e8284d517dd1a99@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: John Matthew Tennant <[email protected]>
@github-actions github-actions Bot added codex-security-review-current The posted Codex security review matches its recorded range. and removed codex-security-review-current The posted Codex security review matches its recorded range. labels Nov 3, 2026

@loganj loganj left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Larry: an independent adversarial review of 05561b4. Source review only: I read the diff, the serve/HEAD/imeta/descriptor-rewrite/bucket-sweep paths it touches, the community-deletion path, and the thumbnail consumers in block/buzz (desktop, mobile, CLI) and block/buzz-app main. I did not run the relay; CI is green on this head.

Approving. The server side is coherent: Thumb is now the one definition of the stored variants, and serve, HEAD, path validation, imeta, the per-tenant descriptor rewrite and the bucket taxonomy all read it. Writes happen before the sidecar, so a failed thumbnail put fails the upload and a retry rewrites both. Community deletion only touches tenant prefixes, so the new global .thumb.webp objects add no new orphan path.

advice: the visible avatar fix still needs a buzz-app change. In buzz-app main, nothing reads the descriptor or imeta thumb. Small avatars come from mediaUrl(..., "small") in src/features/relay/transport.ts, which rewrites any media URL to {sha256}.thumb.jpg. Two allowlists would also block a switch to WebP:

  • media_url in src-tauri/src/relay.rs accepts the extension thumb.jpg only, so the buzz-media proxy returns 403 for .thumb.webp.
  • src/buzz-mcp/node.ts skips the SHA-256 check only for names ending .thumb.jpg, so saving a .thumb.webp fails with "Media did not match its SHA-256".

Mobile never uses thumbnails. Legacy desktop/ is the only client that picks up the new descriptor thumb, through imeta. Until the buzz-app follow-up (WebP first, JPEG fallback for older images) ships, buzz-app users still see the black disc. The PR body says this; a linked follow-up would make the dependency visible to whoever merges.

Comment thread crates/buzz-media/src/thumbnail.rs Resolved
Comment thread crates/buzz-media/Cargo.toml Resolved
@johnmatthewtennant
johnmatthewtennant merged commit f9ef721 into main Nov 4, 2026
160 of 162 checks passed
@johnmatthewtennant
johnmatthewtennant deleted the jtennant/simone/webp-thumbnails branch November 4, 2026 15:11
johnmatthewtennant added a commit to block/buzz-app that referenced this pull request Nov 6, 2026
## Why

Transparent PNG and GIF avatars render on a black background. Relays
always wrote JPEG thumbnails, which have no alpha channel, and the app
requests the small thumbnail for every avatar.

## What

[block/buzz#9003](block/buzz#9003) makes relays
write a `{sha}.thumb.webp` thumbnail that keeps transparency, alongside
the `{sha}.thumb.jpg` that every relay writes.

- Small media URLs now request `.thumb.webp`.
- Both media proxies (Tauri `relay.rs` and the browser
`relay-broker.mjs`) retry `.thumb.jpg` when the relay returns 404.
Relays without the WebP change, and images uploaded before it, keep
rendering as they do today.
- The attachment, Buzz MCP and agent-controller validators accept
`.thumb.webp`. They skip the hash check for it the same way they already
do for `.thumb.jpg`, since thumbnails are derived images.

## Validation

- `webp_thumbnails_fall_back_to_jpeg_only_when_missing` (Tauri) and
"media proxy falls back to the JPEG thumbnail an older relay has"
(browser broker) cover the fallback: the relay returns 404 for
`.thumb.webp`, the proxy requests `.thumb.jpg` next, and the client gets
the JPEG. The Tauri case also checks that a 401 is not retried.
- Both proxies' allowlist tests accept `.thumb.webp` targets.

## Screenshots

Native debug build of this PR's head on macOS, signed in to an isolated
local relay (block/buzz `main`, which writes `.thumb.webp`) as a
dedicated test identity. The avatar is a test fixture: a 512x512 PNG
star with transparent corners.

Before, the same message with the `.thumb.jpg` URL that `main` requests,
served through the native media proxy:

![Before: JPEG thumbnail puts the transparent star avatar on a black
disc](https://github.com/user-attachments/assets/7250841c-6e73-4e27-ad9c-7b9762e3ef8b)

After, this PR requests `.thumb.webp`, and the corners show the app
background:

![After: WebP thumbnail keeps the star avatar
transparent](https://github.com/user-attachments/assets/40e32e5b-9533-40cc-8ce3-5c95d21dd1a5)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: John Tennant <[email protected]>
Co-authored-by: Simone <0828be588d5c9c4d092c5b1cf01761d5aef63d6b104807fd8e8284d517dd1a99@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5.5 <[email protected]>
brow pushed a commit that referenced this pull request Nov 6, 2026
## Summary

Transparent PNG and GIF avatars render on a black background in mobile
avatars and iOS notification avatars. Both load the relay's small
thumbnail, which was always a JPEG with no alpha channel.

[#9003](#9003) makes relays
write a `{sha}.thumb.webp` thumbnail that keeps transparency, alongside
the `{sha}.thumb.jpg` that every relay writes. This PR loads the WebP
first.

- `MediaImage` with `preferThumbnail` tries `.thumb.webp`, then
`.thumb.jpg`, then the original. Only a 404 moves on to the next source;
any other failure still shows the error widget. Missing thumbnails are
remembered as before, so later builds skip straight to the source that
exists.
- `BuzzNotificationAvatarLoader` follows the same order for notification
avatars.

Relays without 9003, and images uploaded before it, keep rendering from
the JPEG as they do today.

### Screenshots

A transparent PNG avatar rendered by `MediaImage` on the same relay,
before and after (widget harness on an iOS simulator).

| Before | After |
| --- | --- |
| ![Before: black square behind the
avatar](https://github.com/user-attachments/assets/5da0a704-df15-4917-9d5c-9745bc83224d)
| ![After: transparent
corners](https://github.com/user-attachments/assets/3d31f4d4-cf65-4923-b367-4fd8cf3af679)
|

### Related issue

AIDA-2308 (internal). The desktop counterpart is
[block/buzz-app#2477](block/buzz-app#2477).

### Testing

- `missing thumbnails fall back to the JPEG, then the original`
(`avatar_image_test.dart`): when the relay 404s every thumbnail, the
avatar requests the WebP, then the JPEG, then the original. When the
WebP fails with a 500, the avatar shows its fallback and requests
neither the JPEG nor the original.
- The relay thumbnail journey tests run against a relay that 404s
`.thumb.webp`, so they cover the older-relay path end to end.
- `BuzzNotificationAvatarLoaderTests` checks the WebP-then-JPEG
thumbnail order for relay media, and that non-relay media gets no
thumbnails.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: John Tennant <[email protected]>
Co-authored-by: Simone <0828be588d5c9c4d092c5b1cf01761d5aef63d6b104807fd8e8284d517dd1a99@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5.5 <[email protected]>
johnmatthewtennant added a commit that referenced this pull request Nov 6, 2026
## Summary

Clients ask for an image's small `.thumb.webp` thumbnail first and fall
back to `.thumb.jpg` on a 404. Images uploaded before
[#9003](#9003) only have the JPEG, so
every client pays an extra 404 for them and transparent avatars keep
showing on a black background. The same thing happens when a thumbnail
write failed at upload time.

When a thumbnail variant is missing but its image original exists, the
relay now regenerates the thumbnails on that request and serves them
with 200. It uses the same generator and storage writes as upload, which
are now one shared `store_image_thumbnails`. Later requests are ordinary
hits.

- GET (full and range) and HEAD both regenerate, so a HEAD-then-GET
client sees the same result.
- Regeneration runs only after the existing tenant, auth and sidecar
checks for that thumbnail.
- It is bounded. At most two regenerations run per relay process
(`try_acquire`, no waiting), and originals larger than `max_image_bytes`
are skipped. In both cases, and for non-images, missing originals or any
decode or storage error, the response is the 404 it is today. Errors are
logged at warn.

### Related issue

Follow-up to [#9003](#9003) and
[#9105](#9105), which made relays
write WebP thumbnails and clients prefer them.

### Testing

Two router-level tests against Postgres and MinIO upload a PNG through
the real route, then delete its thumbnails:

- `missing_thumbnail_is_regenerated_from_image_original`: GET and HEAD
of `.thumb.webp` and `.thumb.jpg` return 200 with the right content
type, and both objects are stored afterward. With regeneration disabled
it fails with `GET webp: 404, expected 200`.
- `missing_thumbnail_is_not_regenerated_when_unsafe`: a non-image blob,
a missing original, an oversize original and a busy relay (no free
permit) each return 404 and store nothing. Once the permits are free,
the same request returns 200. Removing the size cap or the permit guard
fails it.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: John Tennant <[email protected]>
Co-authored-by: Simone <0828be588d5c9c4d092c5b1cf01761d5aef63d6b104807fd8e8284d517dd1a99@buzz.block.builderlab.xyz>
Co-authored-by: Claude Sonnet 5.5 <[email protected]>

This branch was previously deployed

1 inactive deployment
codex-review — 05561b40 Deployed Nov 3, 2026 by johnmatthewtennant via Run Codex Security Review #12182
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex-security-review-current The posted Codex security review matches its recorded range.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants