Skip to content

fix: honour EXIF orientation for uploaded images - #3233

Merged
vpetersson merged 4 commits into
Screenly:masterfrom
vpetersson-bot:fix/exif-orientation-3232
Aug 2, 2026
Merged

fix: honour EXIF orientation for uploaded images#3233
vpetersson merged 4 commits into
Screenly:masterfrom
vpetersson-bot:fix/exif-orientation-3232

Conversation

@vpetersson-bot

Copy link
Copy Markdown
Contributor

Addresses the rotation half of issue 3232 (Autoscale images for display).

Problem

Portrait photos shot on a rotated camera or phone are stored with landscape pixels plus an EXIF Orientation tag (value 6 or 8). Anthias ignored that tag and drew the raw pixels, so those images appeared rotated 90° on the display. The reporter's example is a Canon EOS R6m2 JPEG: 6000×4000 pixels, Orientation = 8.

Root cause

Two independent render paths both dropped the orientation:

  1. Viewer (src/anthias_webview/src/view.cpp): images are decoded with QImage::loadFromData(), which does not apply EXIF orientation. Qt only honours it when decoding through a QImageReader with setAutoTransform(true). This is the path uploaded JPEGs take (the server serves them byte-for-byte).
  2. Server (src/anthias_server/processing.py): the upload-time normalisation pipeline (HEIC / AVIF / TIFF → lossless WebP) converts and saves without applying the tag, and WebP output carries no Orientation tag — so a portrait HEIC (e.g. an iPhone photo) was baked rotated.

There was no EXIF handling anywhere in the codebase.

Fix

  • Viewer: decode via QImageReader with setAutoTransform(true), so the rotation/flip is applied during decode for every format that carries the tag.
  • Server: apply ImageOps.exif_transpose() before the RGBA convert in _convert_image_to_webp(). in_place=True transposes the open image without allocating a second full pixel buffer (respecting the memory discipline already documented there).

Validation

  • Server: new test in tests/test_processing.py feeds a 24×16 landscape source tagged Orientation=6 through _convert_image_to_webp and asserts the WebP comes out 16×24 (upright). It passes with the fix and fails without it (verified by neutralising the transpose line). Full test_processing.py suite: 123 passed. ruff check + ruff format --check clean.
  • Viewer: ran the exact new decode path — QImageReader + setAutoTransform(true) — against the reporter's actual Canon JPEG on Qt 6. The old loadFromData path yields 6000×4000 (landscape, wrong); the new path yields 4000×6000 (portrait, upright). This is pure Qt image I/O, identical behaviour on the Pi's Qt 6.

Scope note

This PR is the orientation fix only. The issue also asks for autoscaling to the display: the viewer already scales images to fit with KeepAspectRatio and a black background (letterboxing), so downscaling large images already works. The remaining question — whether to stop upscaling small images by default, optionally behind a setting — is a separate behaviour change and is intentionally left out of this correctness fix.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HJ3ucEkn62cbgPoisAZ5LQ

Portrait photos shot on a rotated camera/phone are stored with
landscape pixels plus an EXIF Orientation tag (6 or 8). Anthias ignored
that tag and drew the raw pixels, so those images appeared rotated 90°
on screen (issue 3232).

Two independent render paths both dropped the orientation:

- The viewer decoded images with QImage::loadFromData(), which does not
  apply EXIF orientation. Decode through a QImageReader with
  setAutoTransform(true) instead, so the rotation/flip is applied during
  decode for every format that carries the tag (JPEG, PNG, HEIC, ...).
  This covers JPEG, which the server serves byte-for-byte.

- The upload-time normalisation pipeline (HEIC / AVIF / TIFF → lossless
  WebP) converted and saved without applying the tag, and WebP output
  carries no Orientation tag — so those formats were baked rotated.
  Apply ImageOps.exif_transpose() before the RGBA convert; in_place
  avoids allocating a second full pixel buffer.

Validated:
- Server: new test feeds a landscape source tagged Orientation=6 through
  _convert_image_to_webp and asserts the WebP comes out upright
  (transposed dimensions); it fails without the transpose.
- Viewer: QImageReader + setAutoTransform(true) on the reporter's Canon
  EOS R6m2 JPEG (6000x4000, Orientation 8) yields an upright 4000x6000
  image on Qt 6, where the old loadFromData path left it 6000x4000.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HJ3ucEkn62cbgPoisAZ5LQ
@vpetersson-bot
vpetersson-bot requested a review from a team as a code owner August 1, 2026 21:32
@codecov

codecov Bot commented Aug 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (master@d049783). Learn more about missing BASE report.

Additional details and impacted files
@@            Coverage Diff            @@
##             master    #3233   +/-   ##
=========================================
  Coverage          ?   89.44%           
=========================================
  Files             ?       76           
  Lines             ?     8344           
  Branches          ?      892           
=========================================
  Hits              ?     7463           
  Misses            ?      666           
  Partials          ?      215           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vpetersson-bot

Copy link
Copy Markdown
Contributor Author

Testbed validation

Tested on the physical hardware fleet, not just CI.

Server path (processing.py, exif_transpose) — x86 testbed, real pipeline

Uploaded real heavy photos through the actual file_asset → asset-create → celery normalisation flow (not a synthetic fixture):

  • 72 MB TIFF (24 MP) and 9.6 MB HEIC (24 MP) both normalised to an upright 4000×6000 WebP — is_processing cleared, no error_message.
  • No OOM / no crash: celery peaked ~773 MiB / 4.59 GiB, RestartCount=0, OOMKilled=false, dmesg clean.
  • Isolated my change's cost on a genuinely-rotating 24 MP JPEG (stock vs patched, same box): output 6000×4000 (sideways) → 4000×6000 (upright), peak RSS 764 → 780 MiB — i.e. exif_transpose(in_place=True) adds only ~16 MiB.

Viewer path (view.cpp, QImageReader::setAutoTransform)

Ran the exact new decode path against the reporter's actual Canon EOS R6m2 JPEG (6000×4000, Orientation 8) on real Qt 6.11.1:

  • old loadFromData path → 6000×4000 (landscape, sideways — the bug)
  • new QImageReader + setAutoTransform(true)4000×6000 (portrait, upright — fixed)

This is pure Qt image I/O, identical on the Pi's Qt 6. I did not do a full on-device AnthiasViewer image rebuild — both Pi testbeds render via eglfs / headless with no screenshot path, and the decode behaviour is board-independent.

Note surfaced during testing

On a 1 GB Pi 3B+, the existing pipeline already OOM/reboots on a 24 MP image (independent of this change) — fixed separately in #3235.

vpetersson-bot and others added 2 commits August 2, 2026 08:23
SonarCloud's bundled Pillow stubs predate the in_place kwarg (added in
Pillow 9.5; we pin 12.3.0) and flag python:S930 'unexpected named
argument', failing the new-code reliability gate. mypy — which uses the
real Pillow py.typed stubs — accepts it. Annotate the genuine false
positive with # NOSONAR per repo convention.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HJ3ucEkn62cbgPoisAZ5LQ
Split two 'assert a and b' assertions into separate statements
(SonarCloud python:S5906) so a failure names which half broke. Flagged
on this PR's new-code scope after the earlier insertion shifted them in.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HJ3ucEkn62cbgPoisAZ5LQ
@vpetersson-bot

Copy link
Copy Markdown
Contributor Author

Update: verified on-device with a real viewer screenshot

My earlier note said I hadn't rebuilt the viewer on-device — I have now done that on the reporter's exact board (Raspberry Pi 4).

  • Compiled the patched AnthiasViewer against the device's exact Qt (6.8.2, trixie) — built on a Pi 5 (8 GB) since the 1 GB Pi 4 OOMs building qt6-webengine — and swapped just the binary into the running viewer container.
  • Uploaded the reporter's actual photo as the only enabled asset and captured the live eglfs output with ffmpeg -f kmsgrab (the Pi renders straight to DRM/KMS, so grim/fbdev don't apply).

Result on the physical display:

  • stock binary → the photo renders 90° sideways (the bug reproduced on hardware)
  • patched binary → the same photo renders upright, pillarboxed on black

Same asset, same device, only the viewer binary changed. Testbed restored to the shipped image afterwards.

vpetersson
vpetersson previously approved these changes Aug 2, 2026
…ion-3232

# Conflicts:
#	src/anthias_server/processing.py
#	tests/test_processing.py
@sonarqubecloud

sonarqubecloud Bot commented Aug 2, 2026

Copy link
Copy Markdown

@vpetersson
vpetersson merged commit 0a157af into Screenly:master Aug 2, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants