Skip to content

fix(relay): regenerate missing image thumbnails on request - #9174

Merged
johnmatthewtennant merged 1 commit into
mainfrom
jtennant/simone/regenerate-thumbnails
Nov 6, 2026
Merged

johnmatthewtennant merged 1 commit into
mainfrom
jtennant/simone/regenerate-thumbnails

Conversation

@johnmatthewtennant

@johnmatthewtennant johnmatthewtennant commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Clients ask for an image's small .thumb.webp thumbnail first and fall back to .thumb.jpg on a 404. Images uploaded before #9003 only have the JPEG, so every client pays an extra 404 for them and transparent avatars keep showing on a black background. The same thing happens when a thumbnail write failed at upload time.

When a thumbnail variant is missing but its image original exists, the relay now regenerates the thumbnails on that request and serves them with 200. It uses the same generator and storage writes as upload, which are now one shared store_image_thumbnails. Later requests are ordinary hits.

  • GET (full and range) and HEAD both regenerate, so a HEAD-then-GET client sees the same result.
  • Regeneration runs only after the existing tenant, auth and sidecar checks for that thumbnail.
  • It is bounded. At most two regenerations run per relay process (try_acquire, no waiting), and originals larger than max_image_bytes are skipped. In both cases, and for non-images, missing originals or any decode or storage error, the response is the 404 it is today. Errors are logged at warn.

Related issue

Follow-up to #9003 and #9105, which made relays write WebP thumbnails and clients prefer them.

Testing

Two router-level tests against Postgres and MinIO upload a PNG through the real route, then delete its thumbnails:

  • missing_thumbnail_is_regenerated_from_image_original: GET and HEAD of .thumb.webp and .thumb.jpg return 200 with the right content type, and both objects are stored afterward. With regeneration disabled it fails with GET webp: 404, expected 200.
  • missing_thumbnail_is_not_regenerated_when_unsafe: a non-image blob, a missing original, an oversize original and a busy relay (no free permit) each return 404 and store nothing. Once the permits are free, the same request returns 200. Removing the size cap or the permit guard fails it.

🤖 Generated with Claude Code

A thumbnail variant missing from storage returned 404 even when its
image original existed: images uploaded before WebP thumbnails only have
`.thumb.jpg`, and a failed thumbnail write left no retry.

After the existing tenant and sidecar gates, GET and HEAD now rebuild
the thumbnails from the original and serve the requested variant. The
rebuild reuses the upload path's generator through a new shared
`store_image_thumbnails`, runs behind a two-permit `try_acquire`
semaphore, and is skipped for non-images, missing originals, and
originals over `max_image_bytes`. Any skip or failure is a plain 404
(failures log at warn), never a 500.

Co-authored-by: Simone <0828be588d5c9c4d092c5b1cf01761d5aef63d6b104807fd8e8284d517dd1a99@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Signed-off-by: John Tennant <[email protected]>
@johnmatthewtennant
johnmatthewtennant marked this pull request as ready for review November 6, 2026 15:28
@johnmatthewtennant
johnmatthewtennant requested a review from a team as a code owner November 6, 2026 15:28

@wpfleger96 wpfleger96 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Looks good. Regeneration only runs after the Blossom auth, membership and parent-sidecar checks already passed for the thumbnail path, the original key comes from the sidecar's validated ext, and the size cap plus try_acquire keep the cost bounded with a 404 fallback. The upload path is unchanged in behavior after the store_image_thumbnails extraction.

Two nonblocking notes:

  • The two new tests don't run in CI. scripts/check-postgres-test-discovery.py keeps external_infra* modules out of the PostgreSQL lane, and the only MinIO step in _ci-relay.yml ("Admin attachment access-log regression") filters on api::admin::postgres_tests::external_infra_minio_tests::. A sibling step for api::media::tests::postgres_tests::external_infra_minio_tests:: would cover them.
  • regenerate_thumbs decodes the stored original without the upload-time 25 MP pixel check in validate_content. Stored originals already passed it and the image crate's default allocation limit still applies, so this is only defense in depth, but running the imagesize check before storage.get costs little.

@johnmatthewtennant
johnmatthewtennant merged commit 1bb2ad4 into main Nov 6, 2026
96 of 97 checks passed
@johnmatthewtennant
johnmatthewtennant deleted the jtennant/simone/regenerate-thumbnails branch November 6, 2026 16:46
ThePumpingLemma added a commit that referenced this pull request Nov 6, 2026
…l-and-close-stalled-backfill

* origin/main:
  feat(cli): manage sidebar sections with `buzz sidebar sections` (#9191)
  Show mobile floating dates while scrolling (#9185)
  fix(relay): regenerate missing image thumbnails on request (#9174)
  feat(loadgen): model agent turns and sustained holds (#9182)
  fix(cli): list DMs, attach any file, validate kinds, default the relay (#9168)
  Add avatar color backdrop to mobile agent creation (#9184)
  Cache channel memberships once per user (#9060)

Signed-off-by: David Grochowski <[email protected]>
wpfleger96 added a commit that referenced this pull request Nov 6, 2026
…cleanup

* origin/main:
  feat(cli): manage sidebar sections with `buzz sidebar sections` (#9191)
  Show mobile floating dates while scrolling (#9185)
  fix(relay): regenerate missing image thumbnails on request (#9174)
  feat(loadgen): model agent turns and sustained holds (#9182)

Signed-off-by: Will Pfleger <[email protected]>
ThePumpingLemma added a commit that referenced this pull request Nov 6, 2026
…ip-access-when-no-match

* origin/main:
  perf(relay): serve restriction rows for 10 s with cross-pod invalidation (#9199)
  Pin sqlx to the upstream fix for a cancelled BEGIN (#9192)
  feat(cli): manage sidebar sections with `buzz sidebar sections` (#9191)
  Show mobile floating dates while scrolling (#9185)
  fix(relay): regenerate missing image thumbnails on request (#9174)
  feat(loadgen): model agent turns and sustained holds (#9182)
  fix(cli): list DMs, attach any file, validate kinds, default the relay (#9168)
  Add avatar color backdrop to mobile agent creation (#9184)
  Cache channel memberships once per user (#9060)

Signed-off-by: David Grochowski <[email protected]>
bradseiler added a commit that referenced this pull request Nov 6, 2026
* origin/main:
  perf(relay): serve restriction rows for 10 s with cross-pod invalidation (#9199)
  Pin sqlx to the upstream fix for a cancelled BEGIN (#9192)
  feat(cli): manage sidebar sections with `buzz sidebar sections` (#9191)
  Show mobile floating dates while scrolling (#9185)
  fix(relay): regenerate missing image thumbnails on request (#9174)
  feat(loadgen): model agent turns and sustained holds (#9182)
  fix(cli): list DMs, attach any file, validate kinds, default the relay (#9168)

Signed-off-by: Brad Seiler <[email protected]>
johnmatthewtennant added a commit that referenced this pull request Nov 12, 2026
…xels (#9198)

## Summary

Two gaps in the thumbnail regeneration from
[#9174](#9174):

- **Its MinIO tests never ran in CI.**
`api::media::tests::postgres_tests::external_infra_minio_tests` is
excluded from the PostgreSQL lane, which skips `external_infra*`
modules, and the backend integration job's only media MinIO step selects
the admin tests. A sibling step now runs them against the same Postgres
and MinIO.
- **Rebuilding a thumbnail skipped the 25 MP pixel cap.** Upload
validation rejects images over 25 megapixels before they reach the
decoder. A thumbnail rebuilt from a stored original goes straight to
`generate_image_metadata_sync`, so that cap didn't apply. The check is
now a shared `check_image_pixels` that upload validation and thumbnail
generation both run before decoding. On a rebuild, an oversized original
gets the same 404 as any other regeneration failure.

### Related issue

Follow-up to [#9174](#9174), from its
review.

### Testing

- `missing_thumbnail_is_not_regenerated_when_unsafe` now stores a valid
5001x5000 PNG as an image original without thumbnails and requests one
through the media route. On `main` the relay rebuilds and serves it;
with this change the request is a 404 and no thumbnail is written.
- The new CI step runs both `external_infra_minio_tests` cases one at a
time, since they share the fixture image's storage keys.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: John Tennant <[email protected]>
Co-authored-by: Simone <0828be588d5c9c4d092c5b1cf01761d5aef63d6b104807fd8e8284d517dd1a99@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5.5 <[email protected]>

This branch was successfully deployed

No deployments
codex-review — aedb3f1f Deployed Nov 6, 2026 by johnmatthewtennant via Run Codex Security Review #13452
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants