Skip to content

feat: replace picojpeg with JPEGDEC for JPEG image decoding#1136

Merged
daveallie merged 4 commits intocrosspoint-reader:masterfrom
martinbrook:mbrook-jpegdec
Mar 1, 2026
Merged

feat: replace picojpeg with JPEGDEC for JPEG image decoding#1136
daveallie merged 4 commits intocrosspoint-reader:masterfrom
martinbrook:mbrook-jpegdec

Conversation

@martinbrook
Copy link
Contributor

@martinbrook martinbrook commented Feb 23, 2026

Summary

Replaces the picojpeg library with bitbank2/JPEGDEC for JPEG decoding in the EPUB image pipeline. JPEGDEC provides built-in coarse scaling (1/2, 1/4, 1/8), 8-bit grayscale output, and streaming block-based decoding via callbacks.

Includes a pre-build patch script for two JPEGDEC changes affecting progressive JPEG support and EIGHT_BIT_GRAYSCALE mode.

Closes #912

Additional Context

Example progressive jpeg

Good performance increase from JPEGDEC over picojpeg cc @bitbank2 thanks

Baseline JPEG Decode Performance: picojpeg vs JPEGDEC (float in callback) vs JPEGDEC (fixed-point in callback)

Tested with test_jpeg_images.epub on device (ESP32-C3), first decode (no cache).

Image Source Output picojpeg JPEGDEC float JPEGDEC fixed-point vs picojpeg vs float
jpeg_format.jpg 350x250 350x250 313 ms 256 ms 104 ms 3.0x 2.5x
grayscale_test.jpg 400x600 400x600 768 ms 661 ms 246 ms 3.1x 2.7x
gradient_test.jpg 400x500 400x500 707 ms 597 ms 247 ms 2.9x 2.4x
centering_test.jpg 350x400 350x400 502 ms 412 ms 169 ms 3.0x 2.4x
scaling_test.jpg 1200x1500 464x580 5487 ms 1114 ms 668 ms 8.2x 1.7x
wide_scaling_test.jpg 1807x736 464x188 4237 ms 642 ms 497 ms 8.5x 1.3x
cache_test_1.jpg 400x300 400x300 422 ms 348 ms 141 ms 3.0x 2.5x
cache_test_2.jpg 400x300 400x300 424 ms 349 ms 142 ms 3.0x 2.5x

Summary


AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? < PARTIALLY >

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 23, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Replaces PicoJPEG with JPEGDEC: adds per-decode JpegContext, FsFile-based I/O callbacks, heap-allocated JPEGDEC decoder, JPEGDEC draw callback (scaling, progressive support, dithering, optional pixel cache), and a pre-build script to patch JPEGDEC for progressive JPEGs.

Changes

Cohort / File(s) Summary
JPEG decoder implementation
lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp
Replaces PicoJPEG with JPEGDEC. Adds JpegContext, file I/O callbacks (jpegOpen, jpegClose, jpegRead, jpegSeek), heap-managed decoder lifecycle, JPEGDEC-driven dimension/scaling logic, progressive JPEG handling, jpegDrawCallback (downscale/upscale with bilinear), dithering, and optional pixel cache integration.
JPEG decoder header
lib/Epub/Epub/converters/JpegToFramebufferConverter.h
Removes obsolete PicoJPEG private callback declaration; public method signatures (getDimensionsStatic, decodeToFramebuffer) remain unchanged.
Build configuration
platformio.ini
Adds bitbank2/JPEGDEC @ ^1.8.0 to lib_deps and registers scripts/patch_jpegdec.py as a pre-build extra script.
Pre-build patch script
scripts/patch_jpegdec.py
New PlatformIO pre-build script that idempotently patches jpeg.inl in JPEGDEC to handle progressive JPEG cases (AC Huffman table guard and MCU write guard).

Sequence Diagram(s)

sequenceDiagram
    participant App as Application
    participant FS as Filesystem (FsFile)
    participant JPEGDEC as JPEGDEC
    participant Render as RenderCallback/Framebuffer

    App->>FS: jpegOpen(path)
    App->>JPEGDEC: allocate decoder + create JpegContext
    JPEGDEC->>FS: jpegRead / jpegSeek (via callbacks)
    JPEGDEC->>App: jpegDrawCallback(pixel_block)
    App->>Render: render(pixel_block) (scaling, dithering, optional cache)
    JPEGDEC->>App: signal complete
    App->>FS: jpegClose()
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Suggested reviewers

  • daveallie
  • osteotek
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title 'feat: replace picojpeg with JPEGDEC for JPEG image decoding' directly and clearly describes the main change in the PR—migrating from picojpeg to JPEGDEC library.
Linked Issues check ✅ Passed The PR successfully replaces picojpeg with JPEGDEC in the EPUB image pipeline, implementing coarse scaling and 8-bit grayscale support, and includes patches for progressive JPEG handling—directly addressing issue #912.
Out of Scope Changes check ✅ Passed All changes are scoped to the JPEG decoder migration: JpegToFramebufferConverter implementation, header updates, platform configuration (platformio.ini), and the JPEGDEC patching script align with the stated objective.
Description check ✅ Passed The pull request description accurately describes the changeset: replacing picojpeg with JPEGDEC, including details about library features, performance benchmarks, and the patch script for progressive JPEG support.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

Tip

Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs).
Share your feedback on Discord.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp`:
- Around line 55-62: In jpegOpen, avoid unguarded allocation of FsFile via new
FsFile(): check the result of the allocation and handle a failure before using
it; replace the direct new with a guarded allocation (or try/catch for
std::bad_alloc) and if allocation fails return nullptr (and do not call
Storage.openFileForRead or dereference f), otherwise proceed to call
Storage.openFileForRead("JPG", std::string(filename), *f), delete f and return
nullptr on open failure, and on success set *size = f->size() and return f;
refer to the jpegOpen function and the FsFile variable f when making the change.
- Around line 275-325: Detect and short‑circuit when the JPEG reports zero width
or height before any scaling math: after retrieving srcWidth/srcHeight (and
after validateImageDimensions), add a guard that if srcWidth == 0 || srcHeight
== 0 then call jpeg->close(); delete jpeg; and return false. Also ensure any
later divisions use a non‑zero denominator by asserting jpegScaleDenom != 0
before computing ctx.scaledSrcWidth/ctx.scaledSrcHeight and ctx.fineScale (or
handle jpegScaleDenom == 0 by returning false), updating the logic around
chooseJpegScale, isProgressive, and the ctx assignments accordingly.

In `@scripts/patch_jpegdec.py`:
- Around line 54-56: The script currently prints a warning and returns when the
AC-table patch target (checked via the OLD variable against content for
filepath) is not found; change this to fail fast by writing an explicit error to
stderr and exiting non‑zero (e.g. sys.exit(1) or raise RuntimeError) so the
build cannot silently continue unpatched, and apply the identical behavior to
the second patch check (the other OLD/content check around lines 85–88). Allow
an explicit opt‑out by honoring an environment variable (e.g.
PATCH_ALLOW_MISSING or SKIP_PATCH_FAIL) so the exit is skipped when set; include
context in the error message identifying the filepath and that the JPEGDEC
AC-table patch hunk was not found.
- Around line 20-23: The file triggers static analysis errors for SCons-injected
symbols and unused args: add a file-level ruff/flake8 ignore for undefined SCons
symbols (e.g. add a top-of-file comment like "# ruff: noqa: F821, ARG001" or "#
noqa: F821, ARG001") to silence the Import/env undefined symbol warning, and
rename the unused parameters in patch_jpegdec from source, target to _source,
_target (or prefix them with an underscore) to silence ARG001; reference the
Import call, env, and the patch_jpegdec function when making these edits.
- Around line 24-31: Replace the hard-coded libdeps path with a resolved libdeps
directory taken from env.get("PROJECT_LIBDEPS_DIR") falling back to
os.path.join(env["PROJECT_DIR"], ".pio", "libdeps"), and before calling
os.listdir() verify the directory exists with os.path.isdir(); if it doesn't
exist, skip the loop (or log and return) to avoid crashing. Locate the block
that defines libdeps_dir and iterates env_dir (the code that builds jpeg_inl and
calls _apply_ac_table_patch and _apply_mcu_skip_patch) and update it to use the
env fallback and an existence guard so listing only runs when the libdeps
directory is present.

ℹ️ Review info

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 52ca658 and 697eccd.

📒 Files selected for processing (4)
  • lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp
  • lib/Epub/Epub/converters/JpegToFramebufferConverter.h
  • platformio.ini
  • scripts/patch_jpegdec.py
📜 Review details
🧰 Additional context used
🧬 Code graph analysis (1)
lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp (3)
lib/Epub/Epub/converters/DitherUtils.h (2)
  • applyBayerDither4Level (16-28)
  • drawPixelWithRenderMode (31-40)
lib/Epub/Epub/converters/PngToFramebufferConverter.cpp (4)
  • getDimensionsStatic (242-270)
  • getDimensionsStatic (242-242)
  • decodeToFramebuffer (272-392)
  • decodeToFramebuffer (272-273)
lib/Epub/Epub/converters/ImageToFramebufferDecoder.cpp (2)
  • validateImageDimensions (5-12)
  • validateImageDimensions (5-5)
🪛 Ruff (0.15.2)
scripts/patch_jpegdec.py

[error] 20-20: Undefined name Import

(F821)


[warning] 23-23: Unused function argument: source

(ARG001)


[warning] 23-23: Unused function argument: target

(ARG001)


[error] 112-112: Undefined name env

(F821)

🔇 Additional comments (5)
platformio.ini (2)

46-49: Pre-build patch hook is clearly wired.

Keeping the JPEGDEC patch as a pre-build script makes the build-time fix explicit and easy to trace.


60-60: Remove this concern — no version pinning is necessary.

JPEGDEC has no 1.9.x releases (as of February 2026), and the patch script includes idempotent checks and warnings that prevent silent failures. If newer versions are released and the patches fail to apply, the script explicitly warns: "WARNING: JPEGDEC patch target not found — library may have been updated." Keeping ^1.8.0 allows access to bug fixes in 1.8.1–1.8.4 without risk.

Likely an incorrect or invalid review comment.

lib/Epub/Epub/converters/JpegToFramebufferConverter.h (1)

9-20: Public interface remains stable.

Removing the private callback declaration aligns with the JPEGDEC callback flow while keeping the public API unchanged.

lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp (2)

217-243: Dimension probe path looks solid.

Heap checks, open/close, and size extraction are clean and consistent with the PNG path.


365-371: Extension handling is clear and unchanged.

@insunaa
Copy link

insunaa commented Feb 23, 2026

I wish the quality of the progressive jpeg was better, but unfortunately that is an upstream limitation :( Are progressive jpegs just that resource heavy to unpack? A question to ask the jpegdec developer maybe

@martinbrook
Copy link
Contributor Author

martinbrook commented Feb 23, 2026

I wish the quality of the progressive jpeg was better, but unfortunately that is an upstream limitation :( Are progressive jpegs just that resource heavy to unpack? A question to ask the jpegdec developer maybe

As I understand it to do the unpacking one needs to decode the whole image which blows our ram budget. There maybe things I've missed and I've added @bitbank2 to the PR. Pre-processing the images would give the best quality I think but that puts the onus on the user.

@insunaa
Copy link

insunaa commented Feb 23, 2026

yeah, makes sense. cachingpaging all that data to sd card would probably be slow as heck, too

@martinbrook
Copy link
Contributor Author

yeah, makes sense. cachingpaging all that data to sd card would probably be slow as heck, too

There was one other suggestion but again needs discussion with upstream. 1. Change JPEGDEC's progressive support — patch the Huffman table builder to handle 11+ bit AC codes, so it can fully decode progressive JPEGs instead of DC-only. This is the ideal fix but requires deep library modification.

@martinbrook
Copy link
Contributor Author

I wish the quality of the progressive jpeg was better, but unfortunately that is an upstream limitation :( Are progressive jpegs just that resource heavy to unpack? A question to ask the jpegdec developer maybe

The image quality does look better on device than it does in the screenshot above , did you try this branch on your books?

@insunaa
Copy link

insunaa commented Feb 23, 2026

The image quality does look better on device than it does in the screenshot above , did you try this branch on your books?

No, not yet. I'm currently afraid of flashing my X4 due to the platform change that causes some wild preprocessor warnings

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (5)
lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp (2)

54-60: ⚠️ Potential issue | 🟡 Minor

Guard FsFile allocation failure.

new FsFile() can fail under low heap; guard it (e.g., std::nothrow) before dereferencing.

🔧 Suggested change
-  FsFile* f = new FsFile();
-  if (!Storage.openFileForRead("JPG", std::string(filename), *f)) {
+  FsFile* f = new (std::nothrow) FsFile();
+  if (!f || !Storage.openFileForRead("JPG", std::string(filename), *f)) {
     delete f;
     return nullptr;
   }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp` around lines 54 -
60, The code in jpegOpen allocates FsFile with new FsFile() without checking for
allocation failure; change the allocation to use std::nothrow (or otherwise
guard against std::bad_alloc) when creating the FsFile in jpegOpen, check the
pointer for nullptr before using it or calling Storage.openFileForRead, and if
allocation fails return nullptr (ensuring no dereference of a null pointer);
adjust cleanup paths (delete f) accordingly so FsFile is only deleted if
allocation succeeded and after use.

269-319: ⚠️ Potential issue | 🟡 Minor

Add a zero‑dimension guard before scale math.

If a corrupt JPEG reports 0×0, fineScale will divide by zero; bail out early after dimension validation.

🔧 Suggested change
   int srcWidth = jpeg->getWidth();
   int srcHeight = jpeg->getHeight();
 
   if (!validateImageDimensions(srcWidth, srcHeight, "JPEG")) {
     jpeg->close();
     delete jpeg;
     return false;
   }
+  if (srcWidth <= 0 || srcHeight <= 0) {
+    LOG_ERR("JPG", "Invalid JPEG dimensions: %dx%d", srcWidth, srcHeight);
+    jpeg->close();
+    delete jpeg;
+    return false;
+  }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp` around lines 269 -
319, The code can divide by zero when srcWidth or the computed jpegScaleDenom
produce ctx.scaledSrcWidth==0 (e.g., corrupt 0×0 JPEG); after computing
ctx.scaledSrcWidth and ctx.scaledSrcHeight (and before computing ctx.fineScale)
add a guard that treats zero dimensions as a fatal validation failure: close and
delete the jpeg (same cleanup as earlier) and return false. Locate the block
around jpeg->getWidth()/getHeight(), validateImageDimensions, the
jpegScaleOption/denom logic, and the ctx.fineScale assignment to insert this
early bailout.
scripts/patch_jpegdec.py (3)

24-27: ⚠️ Potential issue | 🟡 Minor

Use PROJECT_LIBDEPS_DIR and guard missing directories.

Hard‑coding .pio/libdeps ignores PlatformIO’s configurable libdeps_dir, and os.listdir() will throw if the directory is missing. Prefer PROJECT_LIBDEPS_DIR (with a fallback) and guard existence.

🔧 Suggested change
-    libdeps_dir = os.path.join(env["PROJECT_DIR"], ".pio", "libdeps")
-    for env_dir in os.listdir(libdeps_dir):
+    libdeps_dir = env.get("PROJECT_LIBDEPS_DIR", os.path.join(env["PROJECT_DIR"], ".pio", "libdeps"))
+    if not os.path.isdir(libdeps_dir):
+        return
+    for env_dir in os.listdir(libdeps_dir):
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@scripts/patch_jpegdec.py` around lines 24 - 27, The code currently hard-codes
libdeps_dir and calls os.listdir() without checking existence; update
scripts/patch_jpegdec.py to prefer env.get("PROJECT_LIBDEPS_DIR") with a
fallback to os.path.join(env["PROJECT_DIR"], ".pio", "libdeps"), assign that to
libdeps_dir, then guard with os.path.isdir(libdeps_dir) before calling
os.listdir(libdeps_dir) and handle the missing-dir case (log/raise/exit) so
jpeg_inl path construction (jpeg_inl variable) only runs when the directory
exists.

54-56: ⚠️ Potential issue | 🟠 Major

Fail fast when patch hunks are missing.

A warning allows the build to proceed unpatched, reintroducing the progressive JPEG / MCU_SKIP issues. Prefer a hard failure (optionally with an explicit opt‑out env var).

🔧 Suggested change (apply similarly to both patch checks)
-    if OLD not in content:
-        print("WARNING: JPEGDEC AC table patch target not found in %s — library may have been updated" % filepath)
-        return
+    if OLD not in content:
+        if os.getenv("PATCH_ALLOW_MISSING"):
+            print("WARNING: JPEGDEC AC table patch target not found in %s — library may have been updated" % filepath)
+            return
+        raise RuntimeError(
+            "JPEGDEC AC table patch target not found in %s — verify library version/patches" % filepath
+        )

Also applies to: 85-87

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@scripts/patch_jpegdec.py` around lines 54 - 56, Replace the silent
warning+return when a patch hunk is missing with a hard failure: in the check
"if OLD not in content:" (and the similar second check later) stop the process
with a non‑zero exit (e.g. raise RuntimeError or call sys.exit(1)) so the build
cannot proceed unpatched; allow an explicit opt‑out by checking an env var
(suggest name SKIP_MISSING_PATCH or ALLOW_MISSING_PATCH) and only exit if that
env var is not set. Ensure you import os and sys if needed and update both
occurrences (the "if OLD not in content:" block and the analogous later block)
to follow this pattern.

20-23: ⚠️ Potential issue | 🟡 Minor

Silence Ruff F821/ARG001 for SCons-injected symbols.

Static analysis already flags Import/env as undefined and source/target as unused; add a file‑level ignore and underscore the unused args to keep CI clean.

🔧 Suggested change
+ # ruff: noqa: F821, ARG001
 Import("env")
 import os
 
-def patch_jpegdec(source, target, env):
+def patch_jpegdec(_source, _target, env):
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@scripts/patch_jpegdec.py` around lines 20 - 23, Add a file-level ruff ignore
for the SCons-injected symbols and underscore unused args: add a top-of-file
comment like "ruff: noqa: F821, ARG001" to silence undefined-name and unused-arg
warnings for Import and env, and rename the function parameters in patch_jpegdec
from (source, target, env) to (_source, _target, _env) so the linter knows they
are intentionally unused while keeping the SCons signature intact.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp`:
- Around line 54-60: The code in jpegOpen allocates FsFile with new FsFile()
without checking for allocation failure; change the allocation to use
std::nothrow (or otherwise guard against std::bad_alloc) when creating the
FsFile in jpegOpen, check the pointer for nullptr before using it or calling
Storage.openFileForRead, and if allocation fails return nullptr (ensuring no
dereference of a null pointer); adjust cleanup paths (delete f) accordingly so
FsFile is only deleted if allocation succeeded and after use.
- Around line 269-319: The code can divide by zero when srcWidth or the computed
jpegScaleDenom produce ctx.scaledSrcWidth==0 (e.g., corrupt 0×0 JPEG); after
computing ctx.scaledSrcWidth and ctx.scaledSrcHeight (and before computing
ctx.fineScale) add a guard that treats zero dimensions as a fatal validation
failure: close and delete the jpeg (same cleanup as earlier) and return false.
Locate the block around jpeg->getWidth()/getHeight(), validateImageDimensions,
the jpegScaleOption/denom logic, and the ctx.fineScale assignment to insert this
early bailout.

In `@scripts/patch_jpegdec.py`:
- Around line 24-27: The code currently hard-codes libdeps_dir and calls
os.listdir() without checking existence; update scripts/patch_jpegdec.py to
prefer env.get("PROJECT_LIBDEPS_DIR") with a fallback to
os.path.join(env["PROJECT_DIR"], ".pio", "libdeps"), assign that to libdeps_dir,
then guard with os.path.isdir(libdeps_dir) before calling
os.listdir(libdeps_dir) and handle the missing-dir case (log/raise/exit) so
jpeg_inl path construction (jpeg_inl variable) only runs when the directory
exists.
- Around line 54-56: Replace the silent warning+return when a patch hunk is
missing with a hard failure: in the check "if OLD not in content:" (and the
similar second check later) stop the process with a non‑zero exit (e.g. raise
RuntimeError or call sys.exit(1)) so the build cannot proceed unpatched; allow
an explicit opt‑out by checking an env var (suggest name SKIP_MISSING_PATCH or
ALLOW_MISSING_PATCH) and only exit if that env var is not set. Ensure you import
os and sys if needed and update both occurrences (the "if OLD not in content:"
block and the analogous later block) to follow this pattern.
- Around line 20-23: Add a file-level ruff ignore for the SCons-injected symbols
and underscore unused args: add a top-of-file comment like "ruff: noqa: F821,
ARG001" to silence undefined-name and unused-arg warnings for Import and env,
and rename the function parameters in patch_jpegdec from (source, target, env)
to (_source, _target, _env) so the linter knows they are intentionally unused
while keeping the SCons signature intact.

ℹ️ Review info

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 697eccd and e36a74f.

📒 Files selected for processing (4)
  • lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp
  • lib/Epub/Epub/converters/JpegToFramebufferConverter.h
  • platformio.ini
  • scripts/patch_jpegdec.py
🚧 Files skipped from review as they are similar to previous changes (2)
  • lib/Epub/Epub/converters/JpegToFramebufferConverter.h
  • platformio.ini
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: build
🧰 Additional context used
🪛 Ruff (0.15.2)
scripts/patch_jpegdec.py

[error] 20-20: Undefined name Import

(F821)


[warning] 23-23: Unused function argument: source

(ARG001)


[warning] 23-23: Unused function argument: target

(ARG001)


[error] 112-112: Undefined name env

(F821)

🔇 Additional comments (5)
scripts/patch_jpegdec.py (1)

1-18: Clear module-level documentation.

The docstring concisely explains the two patches and their rationale, which will help future maintainers.

lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp (4)

16-49: Good encapsulation of per‑decode state.

The context struct neatly removes global mutable state and keeps callback usage clean.


97-114: Scale selection logic is clear.

The thresholds map cleanly to JPEGDEC’s scaling options.


116-207: Draw callback bounds handling looks solid.

Stride/size guards and clamping prevent out‑of‑bounds reads while supporting scaling.


211-237: Heap guard + cleanup path look good.

The early heap check and safe decoder allocation keep this path robust.

@Jason2866
Copy link
Contributor

Why patching the library? It is error prone and not needed. Copy the library in the repo folder lib, do the needed changes. Done!

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

♻️ Duplicate comments (6)
lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp (2)

54-62: new FsFile() can throw / crash on low-heap — std::nothrow missing.

Both JPEGDEC allocations below (lines 218, 250) correctly use new (std::nothrow), but jpegOpen uses a throwing new. On ESP32 with exceptions disabled, a failed allocation calls abort() rather than returning nullptr, bypassing the null check entirely.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp` around lines 54 -
62, The jpegOpen function uses throwing new for FsFile which can abort on low
memory; change the allocation to use non-throwing new (std::nothrow) for FsFile
in jpegOpen, check the returned pointer for null and handle failure by not
calling Storage.openFileForRead and returning nullptr (ensuring no leak), and
keep existing delete f on the error path and setting *size only when f is valid;
reference the jpegOpen function and the FsFile allocation to locate the fix.

269-276: Zero-dimension guard before scale math is unverified.

validateImageDimensions may handle zero values, but if it doesn't, ctx.scaledSrcWidth = 0 at line 314 and the division (float)destWidth / ctx.scaledSrcWidth at line 318 would be a divide-by-zero.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp` around lines 269 -
276, The code currently relies on validateImageDimensions to catch zero
dimensions but then uses srcWidth/srcHeight to compute ctx.scaledSrcWidth and
perform division (e.g., (float)destWidth / ctx.scaledSrcWidth), risking a
divide-by-zero; add an explicit guard after int srcWidth = jpeg->getWidth(); int
srcHeight = jpeg->getHeight(); to check srcWidth > 0 && srcHeight > 0 and on
failure call jpeg->close(); delete jpeg; and return false (or propagate an
error) before any scaling math, referencing validateImageDimensions and
ctx.scaledSrcWidth to ensure the division cannot occur with a zero denominator.
scripts/patch_jpegdec.py (4)

54-56: Warn-and-continue allows a silently unpatched build.

If the AC-table patch target is absent, the build continues with the unpatched library, reintroducing the progressive JPEG crash. Consider raise RuntimeError(...) to fail fast.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@scripts/patch_jpegdec.py` around lines 54 - 56, The current check using "if
OLD not in content" only prints a warning and returns, allowing a silently
unpatched build; change this to raise a RuntimeError (including filepath and a
clear message) so the script fails fast when the AC-table patch target (OLD) is
missing from content, ensuring the build stops instead of proceeding with an
unpatched library.

20-23: Suppress Ruff F821/ARG001 for SCons-injected globals and unused arguments.

Import and env are SCons-injected globals that static analysis cannot resolve, and source/target are unused in patch_jpegdec.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@scripts/patch_jpegdec.py` around lines 20 - 23, Add a file-level Ruff
suppression so SCons-injected globals and intentionally unused args are ignored:
insert a top-line directive like "# ruff: noqa: F821, ARG001" above the existing
Import("env") line; keep the Import("env") invocation and the
patch_jpegdec(source, target, env) signature unchanged so SCons behavior
remains, but the linter will no longer flag undefined name Import/env (F821) or
the unused parameters source/target (ARG001).

24-31: Use PROJECT_LIBDEPS_DIR and guard against a missing directory.

Hard-coding .pio/libdeps ignores PlatformIO's configurable libdeps_dir. Additionally, os.listdir() on a non-existent path crashes the build.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@scripts/patch_jpegdec.py` around lines 24 - 31, Replace the hard-coded
libdeps path and add a directory existence check: use
env.get("PROJECT_LIBDEPS_DIR", os.path.join(env["PROJECT_DIR"], ".pio",
"libdeps")) to set libdeps_dir, then guard with if not
os.path.isdir(libdeps_dir): return (or log and exit) before calling os.listdir;
keep the same logic that iterates env_dir and calls
_apply_ac_table_patch(jpeg_inl) and _apply_mcu_skip_patch(jpeg_inl) when
jpeg.inl is found so the functions (_apply_ac_table_patch,
_apply_mcu_skip_patch) and variable names remain unchanged.

85-88: Same warn-and-continue issue for the MCU_SKIP patch.

Same as above – a missing OLD_DC target silently skips patching the JPEGDecodeMCU_P guard, leaving the wild-pointer write intact.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@scripts/patch_jpegdec.py` around lines 85 - 88, The current MCU_SKIP patch
check quietly returns when OLD_DC is missing, which can leave the
JPEGDecodeMCU_P guard unpatched; instead, when OLD_DC is not found in content,
stop processing and fail loudly—replace the silent return with an error exit or
raise (e.g., call sys.exit(1) or raise RuntimeError) after printing a clear
error message that includes filepath and mentions the MCU_SKIP/JPEGDecodeMCU_P
patch, so the CI/dev knows the patch did not apply.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp`:
- Line 7: Remove the stray null preprocessor directive ("#") that appears
between the include lines in JpegToFramebufferConverter.cpp; open the file and
delete the lone "#" found between the `#include` <Logging.h> and `#include`
<cstdlib> so the include block is clean and no unintended null directive
remains.
- Around line 287-318: The useExactDimensions branch computes targetScale using
only width (targetScale = destWidth/srcWidth) which causes chooseJpegScale and
ctx.fineScale to be width-only and leads to vertical cropping in
jpegDrawCallback; update the logic in JpegToFramebufferConverter.cpp (and the
identical code in PngToFramebufferConverter.cpp) to compute targetScale =
min((float)destWidth/srcWidth, (float)destHeight/srcHeight) and then fix the
single-axis fineScale: either add fineScaleX and fineScaleY to JpegContext and
propagate them into jpegDrawCallback (and use both to scale X/Y), or adjust
destWidth/destHeight to preserve the chosen aspect ratio before computing
ctx.fineScale so a single uniform fineScale is correct; ensure chooseJpegScale
still receives the corrected targetScale.
- Around line 76-90: jpegRead and jpegSeek must guard against FsFile I/O
failures and only update pFile->iPos on success: in jpegRead check
FsFile::read() return value and if it is negative return 0 (do not change
pFile->iPos); if non-negative add that value to pFile->iPos and return it; in
jpegSeek check the boolean result of FsFile::seek(), return -1 and leave
pFile->iPos unchanged on failure, and only update pFile->iPos to the requested
pos and return pos on success so JPEGDEC sees accurate positions.

---

Duplicate comments:
In `@lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp`:
- Around line 54-62: The jpegOpen function uses throwing new for FsFile which
can abort on low memory; change the allocation to use non-throwing new
(std::nothrow) for FsFile in jpegOpen, check the returned pointer for null and
handle failure by not calling Storage.openFileForRead and returning nullptr
(ensuring no leak), and keep existing delete f on the error path and setting
*size only when f is valid; reference the jpegOpen function and the FsFile
allocation to locate the fix.
- Around line 269-276: The code currently relies on validateImageDimensions to
catch zero dimensions but then uses srcWidth/srcHeight to compute
ctx.scaledSrcWidth and perform division (e.g., (float)destWidth /
ctx.scaledSrcWidth), risking a divide-by-zero; add an explicit guard after int
srcWidth = jpeg->getWidth(); int srcHeight = jpeg->getHeight(); to check
srcWidth > 0 && srcHeight > 0 and on failure call jpeg->close(); delete jpeg;
and return false (or propagate an error) before any scaling math, referencing
validateImageDimensions and ctx.scaledSrcWidth to ensure the division cannot
occur with a zero denominator.

In `@scripts/patch_jpegdec.py`:
- Around line 54-56: The current check using "if OLD not in content" only prints
a warning and returns, allowing a silently unpatched build; change this to raise
a RuntimeError (including filepath and a clear message) so the script fails fast
when the AC-table patch target (OLD) is missing from content, ensuring the build
stops instead of proceeding with an unpatched library.
- Around line 20-23: Add a file-level Ruff suppression so SCons-injected globals
and intentionally unused args are ignored: insert a top-line directive like "#
ruff: noqa: F821, ARG001" above the existing Import("env") line; keep the
Import("env") invocation and the patch_jpegdec(source, target, env) signature
unchanged so SCons behavior remains, but the linter will no longer flag
undefined name Import/env (F821) or the unused parameters source/target
(ARG001).
- Around line 24-31: Replace the hard-coded libdeps path and add a directory
existence check: use env.get("PROJECT_LIBDEPS_DIR",
os.path.join(env["PROJECT_DIR"], ".pio", "libdeps")) to set libdeps_dir, then
guard with if not os.path.isdir(libdeps_dir): return (or log and exit) before
calling os.listdir; keep the same logic that iterates env_dir and calls
_apply_ac_table_patch(jpeg_inl) and _apply_mcu_skip_patch(jpeg_inl) when
jpeg.inl is found so the functions (_apply_ac_table_patch,
_apply_mcu_skip_patch) and variable names remain unchanged.
- Around line 85-88: The current MCU_SKIP patch check quietly returns when
OLD_DC is missing, which can leave the JPEGDecodeMCU_P guard unpatched; instead,
when OLD_DC is not found in content, stop processing and fail loudly—replace the
silent return with an error exit or raise (e.g., call sys.exit(1) or raise
RuntimeError) after printing a clear error message that includes filepath and
mentions the MCU_SKIP/JPEGDecodeMCU_P patch, so the CI/dev knows the patch did
not apply.

ℹ️ Review info

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e36a74f and e541cfb.

📒 Files selected for processing (4)
  • lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp
  • lib/Epub/Epub/converters/JpegToFramebufferConverter.h
  • platformio.ini
  • scripts/patch_jpegdec.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • platformio.ini
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: build
  • GitHub Check: cppcheck
🧰 Additional context used
🪛 Ruff (0.15.2)
scripts/patch_jpegdec.py

[error] 20-20: Undefined name Import

(F821)


[warning] 23-23: Unused function argument: source

(ARG001)


[warning] 23-23: Unused function argument: target

(ARG001)


[error] 112-112: Undefined name env

(F821)

🔇 Additional comments (1)
lib/Epub/Epub/converters/JpegToFramebufferConverter.h (1)

9-21: LGTM – clean removal of the private PicoJPEG callback.

The public API surface is fully preserved and the removed jpegReadCallback is correctly superseded by the free-function I/O callbacks (jpegOpen/jpegClose/jpegRead/jpegSeek) in the .cpp.

@martinbrook
Copy link
Contributor Author

Why patching the library? It is error prone and not needed. Copy the library in the repo folder lib, do the needed changes. Done!

I'd like to see if upstream can help out on this , I wouldn't want to carry the whole library here.

@Jason2866
Copy link
Contributor

I'd like to see if upstream can help out on this , I wouldn't want to carry the whole library here.

Best is always if changes are done and accepted upstream. Imho patching is a bad approach if changes are not in upstream. A few kbytes is less pain than maintaining patch errors. Furthermore code is obfuscated

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (5)
lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp (4)

5-9: Remove the stray null preprocessor directive.
The lone # is likely an accidental artifact and should be removed.

🧹 Proposed fix
 `#include` <JPEGDEC.h>
 `#include` <Logging.h>
-#
 `#include` <cstdlib>
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp` around lines 5 - 9,
The top of JpegToFramebufferConverter.cpp has a stray lone preprocessor token
causing a compile error; remove the single `#` line between the include block
(near the includes for JPEGDEC and Logging and the stdlib/new includes) so only
valid include directives remain (e.g., keep `#include` <JPEGDEC.h>, `#include`
<Logging.h>, `#include` <cstdlib>, `#include` <new> and delete the stray `#`).

76-89: ⚠️ Potential issue | 🟡 Minor

Don’t update iPos on failed read/seek.
If read() returns an error or seek() fails, iPos becomes inconsistent with the real file position.

🔧 Proposed fix
 int32_t bytesRead = f->read(pBuf, len);
-  pFile->iPos += bytesRead;
-  return bytesRead;
+  if (bytesRead > 0) pFile->iPos += bytesRead;
+  else bytesRead = 0;
+  return bytesRead;
 }
 
 int32_t jpegSeek(JPEGFILE* pFile, int32_t pos) {
   FsFile* f = reinterpret_cast<FsFile*>(pFile->fHandle);
   if (!f) return -1;
-  f->seek(pos);
-  pFile->iPos = pos;
-  return pos;
+  if (!f->seek(pos)) return -1;
+  pFile->iPos = pos;
+  return pos;
 }
#!/bin/bash
# Verify FsFile read/seek return contracts in the repo
rg -nP --type=cpp --type=h -C2 '\bclass\s+FsFile\b'
rg -nP --type=cpp --type=h -C2 '\bFsFile::read\s*\('
rg -nP --type=cpp --type=h -C2 '\bFsFile::seek\s*\('
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp` around lines 76 -
89, The jpegRead and jpegSeek helpers update pFile->iPos unconditionally which
can desync state on errors; change jpegRead (function jpegRead) to call
FsFile::read, check its return value and only add to pFile->iPos when bytesRead
> 0 (handle zero/negative as errors and return as-is), and change jpegSeek
(function jpegSeek) to inspect the return from FsFile::seek (use its
success/position return contract) and only set pFile->iPos when seek succeeded;
do not modify iPos on failure and propagate the error/return value from the
underlying FsFile::read/seek instead.

54-61: ⚠️ Potential issue | 🟡 Minor

Guard FsFile allocation failure in jpegOpen.
If allocation fails, dereferencing f can crash on low heap.

🔧 Proposed fix
-  FsFile* f = new FsFile();
-  if (!Storage.openFileForRead("JPG", std::string(filename), *f)) {
+  FsFile* f = new (std::nothrow) FsFile();
+  if (!f || !Storage.openFileForRead("JPG", std::string(filename), *f)) {
     delete f;
     return nullptr;
   }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp` around lines 54 -
61, The function jpegOpen currently assumes FsFile allocation succeeds and
immediately dereferences f; change allocation to use std::nothrow (FsFile* f =
new (std::nothrow) FsFile()) and check for nullptr before calling
Storage.openFileForRead or dereferencing f; if allocation fails return nullptr
(and set *size to 0 or leave unchanged per convention). Ensure the null-check is
placed before the call to Storage.openFileForRead("JPG", std::string(filename),
*f) and keep existing cleanup (delete f) and return paths intact in jpegOpen.

294-326: ⚠️ Potential issue | 🟠 Major

useExactDimensions scales by width only, causing cropping.
Using width-only targetScale and fineScale can silently crop when height is the limiting axis.

⚠️ Partial fix (scale selection only)
-    targetScale = (float)destWidth / srcWidth;
+    float scaleX = (float)destWidth / srcWidth;
+    float scaleY = (float)destHeight / srcHeight;
+    targetScale = (scaleX < scaleY) ? scaleX : scaleY;

A full fix still needs fineScale to respect both axes (e.g., separate X/Y scales or aspect-preserving dest sizing).

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp` around lines 294 -
326, The current useExactDimensions branch picks targetScale from width only,
which can crop vertically; update the logic so targetScale and the fine scaling
respect both axes: when useExactDimensions is true, compute destWidth and
destHeight from config.maxWidth/maxHeight but derive separate fineScaleX =
(float)destWidth / ctx.scaledSrcWidth and fineScaleY = (float)destHeight /
ctx.scaledSrcHeight (or compute ctx.fineScale = min(fineScaleX, fineScaleY) and
adjust destWidth/destHeight to preserve aspect) and store these on ctx (e.g.,
ctx.fineScaleX/ctx.fineScaleY or set ctx.fineScale using the min) instead of
using a width-only targetScale; keep the JPEG built-in scaling selection
(isProgressive, chooseJpegScale, JPEG_SCALE_EIGHTH) the same and compute
ctx.scaledSrcWidth/ctx.scaledSrcHeight before deriving the per-axis fine scales.
scripts/patch_jpegdec.py (1)

23-33: ⚠️ Potential issue | 🟡 Minor

Avoid hard-coded .pio/libdeps to respect PlatformIO config.
This path can be customized, so use PROJECT_LIBDEPS_DIR with a fallback.

🔧 Proposed fix
-    libdeps_dir = os.path.join(env["PROJECT_DIR"], ".pio", "libdeps")
+    libdeps_dir = env.get("PROJECT_LIBDEPS_DIR", os.path.join(env["PROJECT_DIR"], ".pio", "libdeps"))
     if not os.path.isdir(libdeps_dir):
         return
#!/bin/bash
# Check for custom libdeps_dir usage and existing PROJECT_LIBDEPS_DIR references
rg -n "libdeps_dir|PROJECT_LIBDEPS_DIR" platformio.ini
rg -n "PROJECT_LIBDEPS_DIR" -g '*.py'
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@scripts/patch_jpegdec.py` around lines 23 - 33, The code in patch_jpegdec
currently hardcodes libdeps_dir to os.path.join(env["PROJECT_DIR"], ".pio",
"libdeps"); change it to prefer env["PROJECT_LIBDEPS_DIR"] if set and only fall
back to os.path.join(env["PROJECT_DIR"], ".pio", "libdeps") so PlatformIO
customizations are respected. Update the patch_jpegdec function to obtain
libdeps_dir = env.get("PROJECT_LIBDEPS_DIR", os.path.join(env["PROJECT_DIR"],
".pio", "libdeps")), keep the existing isdir check and loop that looks for
jpeg_inl and invokes _apply_ac_table_patch and _apply_mcu_skip_patch; ensure
behavior is unchanged when the directory is absent.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp`:
- Around line 5-9: The top of JpegToFramebufferConverter.cpp has a stray lone
preprocessor token causing a compile error; remove the single `#` line between
the include block (near the includes for JPEGDEC and Logging and the stdlib/new
includes) so only valid include directives remain (e.g., keep `#include`
<JPEGDEC.h>, `#include` <Logging.h>, `#include` <cstdlib>, `#include` <new> and delete
the stray `#`).
- Around line 76-89: The jpegRead and jpegSeek helpers update pFile->iPos
unconditionally which can desync state on errors; change jpegRead (function
jpegRead) to call FsFile::read, check its return value and only add to
pFile->iPos when bytesRead > 0 (handle zero/negative as errors and return
as-is), and change jpegSeek (function jpegSeek) to inspect the return from
FsFile::seek (use its success/position return contract) and only set pFile->iPos
when seek succeeded; do not modify iPos on failure and propagate the
error/return value from the underlying FsFile::read/seek instead.
- Around line 54-61: The function jpegOpen currently assumes FsFile allocation
succeeds and immediately dereferences f; change allocation to use std::nothrow
(FsFile* f = new (std::nothrow) FsFile()) and check for nullptr before calling
Storage.openFileForRead or dereferencing f; if allocation fails return nullptr
(and set *size to 0 or leave unchanged per convention). Ensure the null-check is
placed before the call to Storage.openFileForRead("JPG", std::string(filename),
*f) and keep existing cleanup (delete f) and return paths intact in jpegOpen.
- Around line 294-326: The current useExactDimensions branch picks targetScale
from width only, which can crop vertically; update the logic so targetScale and
the fine scaling respect both axes: when useExactDimensions is true, compute
destWidth and destHeight from config.maxWidth/maxHeight but derive separate
fineScaleX = (float)destWidth / ctx.scaledSrcWidth and fineScaleY =
(float)destHeight / ctx.scaledSrcHeight (or compute ctx.fineScale =
min(fineScaleX, fineScaleY) and adjust destWidth/destHeight to preserve aspect)
and store these on ctx (e.g., ctx.fineScaleX/ctx.fineScaleY or set ctx.fineScale
using the min) instead of using a width-only targetScale; keep the JPEG built-in
scaling selection (isProgressive, chooseJpegScale, JPEG_SCALE_EIGHTH) the same
and compute ctx.scaledSrcWidth/ctx.scaledSrcHeight before deriving the per-axis
fine scales.

In `@scripts/patch_jpegdec.py`:
- Around line 23-33: The code in patch_jpegdec currently hardcodes libdeps_dir
to os.path.join(env["PROJECT_DIR"], ".pio", "libdeps"); change it to prefer
env["PROJECT_LIBDEPS_DIR"] if set and only fall back to
os.path.join(env["PROJECT_DIR"], ".pio", "libdeps") so PlatformIO customizations
are respected. Update the patch_jpegdec function to obtain libdeps_dir =
env.get("PROJECT_LIBDEPS_DIR", os.path.join(env["PROJECT_DIR"], ".pio",
"libdeps")), keep the existing isdir check and loop that looks for jpeg_inl and
invokes _apply_ac_table_patch and _apply_mcu_skip_patch; ensure behavior is
unchanged when the directory is absent.

ℹ️ Review info

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e541cfb and b02c21e.

📒 Files selected for processing (4)
  • lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp
  • lib/Epub/Epub/converters/JpegToFramebufferConverter.h
  • platformio.ini
  • scripts/patch_jpegdec.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • platformio.ini
📜 Review details
🧰 Additional context used
🧠 Learnings (4)
📓 Common learnings
Learnt from: martinbrook
Repo: crosspoint-reader/crosspoint-reader PR: 1136
File: scripts/patch_jpegdec.py:54-56
Timestamp: 2026-02-23T19:57:33.257Z
Learning: In the crosspoint-reader project, the patch_jpegdec.py pre-build script uses warn-and-continue (print WARNING and return) rather than fail-fast when patch targets are not found in the JPEGDEC library. This approach is preferred because the patches address edge cases (progressive JPEG AC tables, MCU_SKIP crashes), and a missing target likely indicates either an upstream fix or a library version change requiring manual patch updates. Failing the build would block unrelated development unnecessarily.
Learnt from: martinbrook
Repo: crosspoint-reader/crosspoint-reader PR: 1136
File: scripts/patch_jpegdec.py:20-23
Timestamp: 2026-02-23T19:55:03.054Z
Learning: PlatformIO pre-build scripts in the crosspoint-reader project use the standard SCons pattern `Import("env")` and callback signature `def patch_jpegdec(source, target, env):` without noqa comments or underscore-prefixing unused parameters, following the convention used in other scripts in the project.
📚 Learning: 2026-02-23T19:55:03.054Z
Learnt from: martinbrook
Repo: crosspoint-reader/crosspoint-reader PR: 1136
File: scripts/patch_jpegdec.py:20-23
Timestamp: 2026-02-23T19:55:03.054Z
Learning: PlatformIO pre-build scripts in the crosspoint-reader project use the standard SCons pattern `Import("env")` and callback signature `def patch_jpegdec(source, target, env):` without noqa comments or underscore-prefixing unused parameters, following the convention used in other scripts in the project.

Applied to files:

  • scripts/patch_jpegdec.py
📚 Learning: 2026-02-23T19:57:33.257Z
Learnt from: martinbrook
Repo: crosspoint-reader/crosspoint-reader PR: 1136
File: scripts/patch_jpegdec.py:54-56
Timestamp: 2026-02-23T19:57:33.257Z
Learning: In the crosspoint-reader project, the patch_jpegdec.py pre-build script uses warn-and-continue (print WARNING and return) rather than fail-fast when patch targets are not found in the JPEGDEC library. This approach is preferred because the patches address edge cases (progressive JPEG AC tables, MCU_SKIP crashes), and a missing target likely indicates either an upstream fix or a library version change requiring manual patch updates. Failing the build would block unrelated development unnecessarily.

Applied to files:

  • scripts/patch_jpegdec.py
  • lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp
📚 Learning: 2026-02-23T19:55:03.054Z
Learnt from: martinbrook
Repo: crosspoint-reader/crosspoint-reader PR: 1136
File: scripts/patch_jpegdec.py:20-23
Timestamp: 2026-02-23T19:55:03.054Z
Learning: In the crosspoint-reader project, enable Python linters (e.g., Ruff or Flake8) in CI for all Python scripts. Run linting as part of the CI workflow and fix reported issues to maintain code quality. Do not skip linting for any Python files.

Applied to files:

  • scripts/patch_jpegdec.py
🧬 Code graph analysis (1)
lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp (4)
lib/Epub/Epub/converters/DitherUtils.h (2)
  • applyBayerDither4Level (16-28)
  • drawPixelWithRenderMode (31-40)
lib/Epub/Epub/converters/PngToFramebufferConverter.cpp (4)
  • getDimensionsStatic (241-269)
  • getDimensionsStatic (241-241)
  • decodeToFramebuffer (271-391)
  • decodeToFramebuffer (271-272)
src/main.cpp (1)
  • renderer (39-39)
lib/Epub/Epub/converters/ImageToFramebufferDecoder.cpp (2)
  • validateImageDimensions (5-12)
  • validateImageDimensions (5-5)
🪛 Ruff (0.15.2)
scripts/patch_jpegdec.py

[error] 20-20: Undefined name Import

(F821)


[warning] 23-23: Unused function argument: source

(ARG001)


[warning] 23-23: Unused function argument: target

(ARG001)


[error] 114-114: Undefined name env

(F821)

🔇 Additional comments (9)
lib/Epub/Epub/converters/JpegToFramebufferConverter.h (1)

9-20: Header change looks good.
Public API remains stable and the interface is clean.

lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp (4)

97-114: Coarse-scale selection is clear and correct.
The thresholds map cleanly to JPEGDEC’s built-in scale options.


116-207: Draw callback implementation looks solid.
Bilinear upscale handling and dithering integration are cleanly structured.


211-237: Dimension probe path looks good.
Heap checks and decoder lifecycle are handled safely.


366-371: Format support check looks good.
Extension normalization is straightforward and correct.

scripts/patch_jpegdec.py (4)

1-18: Docstring is clear and useful.
The patch rationale and safety notes are well explained.


34-64: AC-table patch looks good.
Idempotent marker and replacement logic are clean.


65-112: MCU_SKIP guard patch looks solid.
The two-stage patching is clear and safe.


114-114: Pre-action registration is straightforward.
Wiring the script to buildprog is clear.

@martinbrook
Copy link
Contributor Author

martinbrook commented Feb 23, 2026

I'd like to see if upstream can help out on this , I wouldn't want to carry the whole library here.

Best is always if changes are done and accepted upstream. Imho patching is a bad approach if changes are not in upstream. A few kbytes is less pain than maintaining patch errors. Furthermore code is obfuscated

I agree so was keen to talk to upstream as I mention in the PR

Replaces the picojpeg library with bitbank2/JPEGDEC for JPEG decoding
in the EPUB image pipeline. JPEGDEC provides built-in coarse scaling
(1/2, 1/4, 1/8), 8-bit grayscale output, and streaming block-based
decoding via callbacks.

Draw callback iterates destination pixels (matching the PNG decoder
approach) with bilinear interpolation for upscale to smooth progressive
JPEG DC-only block boundaries.

Includes a pre-build patch script for two JPEGDEC bugs affecting
progressive JPEG support and EIGHT_BIT_GRAYSCALE mode.
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (3)
lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp (2)

54-61: ⚠️ Potential issue | 🟡 Minor

Guard FsFile allocation failure. Line 55 uses new FsFile() without handling allocation failure; on low heap this can throw or return null depending on build flags.

🔧 Proposed change
-  FsFile* f = new FsFile();
-  if (!Storage.openFileForRead("JPG", std::string(filename), *f)) {
+  FsFile* f = new (std::nothrow) FsFile();
+  if (!f || !Storage.openFileForRead("JPG", std::string(filename), *f)) {
     delete f;
     return nullptr;
   }

You can confirm how new behaves in this build by checking exception flags:

#!/bin/bash
rg -n "fno-exceptions|fexceptions" -g "*.ini" -g "*.txt" -g "*.mk" -g "*.cfg"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp` around lines 54 -
61, The FsFile allocation in jpegOpen is not guarded: change the allocation to
handle allocation failure by either using std::nothrow (e.g., new (std::nothrow)
FsFile()) and checking for nullptr, or wrap the allocation and subsequent
Storage.openFileForRead call in a try/catch to catch std::bad_alloc; if
allocation fails return nullptr and avoid calling Storage.openFileForRead.
Update jpegOpen to validate the FsFile pointer (or catch the exception) before
dereferencing and ensure cleanup on failure.

295-327: ⚠️ Potential issue | 🟠 Major

useExactDimensions uses width-only scale, which can crop vertically. Line 298 computes targetScale from width only, while fineScale is uniform; if height is the tighter constraint, the render path silently crops instead of scaling to fit.

🔧 Directional fix (partial) + follow-up
-    targetScale = (float)destWidth / srcWidth;
+    float tScaleX = (float)destWidth / srcWidth;
+    float tScaleY = (float)destHeight / srcHeight;
+    targetScale = (tScaleX < tScaleY) ? tScaleX : tScaleY;

To fully resolve, either (a) compute separate fineScaleX/fineScaleY, or (b) adjust destWidth/destHeight to preserve the chosen aspect ratio so a single fineScale is valid.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp` around lines 295 -
327, The useExactDimensions branch sets destWidth/destHeight to
config.maxWidth/config.maxHeight but computes targetScale using width only,
which can crop vertically; update the logic in the
JpegToFramebufferConverter.cpp block (the branch guarded by
config.useExactDimensions) to either compute separate fineScaleX/fineScaleY and
store both on ctx (e.g., ctx.fineScaleX and ctx.fineScaleY) and downstream use
them, or compute a uniform targetScale from the tighter of width/height
constraints and then adjust destWidth or destHeight to preserve aspect ratio
before computing ctx.fineScale; reference the variables
config.useExactDimensions, destWidth, destHeight, targetScale,
ctx.scaledSrcWidth/Height and ctx.fineScale when making the change.
scripts/patch_jpegdec.py (1)

23-30: ⚠️ Potential issue | 🟡 Minor

Honor PlatformIO’s configurable libdeps_dir. Line 25 hard‑codes .pio/libdeps, so patches won’t run if libdeps_dir is customized. Consider using PROJECT_LIBDEPS_DIR with a fallback.

🔧 Proposed change
-    libdeps_dir = os.path.join(env["PROJECT_DIR"], ".pio", "libdeps")
+    libdeps_dir = env.get("PROJECT_LIBDEPS_DIR",
+                          os.path.join(env["PROJECT_DIR"], ".pio", "libdeps"))
     if not os.path.isdir(libdeps_dir):
         return

To verify whether libdeps_dir is customized in this repo, you can run:

#!/bin/bash
rg -n "libdeps_dir" -g "platformio.ini" -g "*.ini"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@scripts/patch_jpegdec.py` around lines 23 - 30, The patch_jpegdec function
currently hardcodes libdeps_dir to os.path.join(env["PROJECT_DIR"], ".pio",
"libdeps"); change it to honor PlatformIO's configurable libdeps_dir by reading
env["PROJECT_LIBDEPS_DIR"] with a sensible fallback to the existing path. Update
the libdeps_dir assignment in patch_jpegdec to prefer
env.get("PROJECT_LIBDEPS_DIR") (or similar) and only fall back to
os.path.join(env["PROJECT_DIR"], ".pio", "libdeps") if that key is missing, so
the subsequent logic that looks for JPEGDEC/src/jpeg.inl works with customized
setups.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp`:
- Around line 54-61: The FsFile allocation in jpegOpen is not guarded: change
the allocation to handle allocation failure by either using std::nothrow (e.g.,
new (std::nothrow) FsFile()) and checking for nullptr, or wrap the allocation
and subsequent Storage.openFileForRead call in a try/catch to catch
std::bad_alloc; if allocation fails return nullptr and avoid calling
Storage.openFileForRead. Update jpegOpen to validate the FsFile pointer (or
catch the exception) before dereferencing and ensure cleanup on failure.
- Around line 295-327: The useExactDimensions branch sets destWidth/destHeight
to config.maxWidth/config.maxHeight but computes targetScale using width only,
which can crop vertically; update the logic in the
JpegToFramebufferConverter.cpp block (the branch guarded by
config.useExactDimensions) to either compute separate fineScaleX/fineScaleY and
store both on ctx (e.g., ctx.fineScaleX and ctx.fineScaleY) and downstream use
them, or compute a uniform targetScale from the tighter of width/height
constraints and then adjust destWidth or destHeight to preserve aspect ratio
before computing ctx.fineScale; reference the variables
config.useExactDimensions, destWidth, destHeight, targetScale,
ctx.scaledSrcWidth/Height and ctx.fineScale when making the change.

In `@scripts/patch_jpegdec.py`:
- Around line 23-30: The patch_jpegdec function currently hardcodes libdeps_dir
to os.path.join(env["PROJECT_DIR"], ".pio", "libdeps"); change it to honor
PlatformIO's configurable libdeps_dir by reading env["PROJECT_LIBDEPS_DIR"] with
a sensible fallback to the existing path. Update the libdeps_dir assignment in
patch_jpegdec to prefer env.get("PROJECT_LIBDEPS_DIR") (or similar) and only
fall back to os.path.join(env["PROJECT_DIR"], ".pio", "libdeps") if that key is
missing, so the subsequent logic that looks for JPEGDEC/src/jpeg.inl works with
customized setups.

ℹ️ Review info

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b02c21e and 2451fe3.

📒 Files selected for processing (4)
  • lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp
  • lib/Epub/Epub/converters/JpegToFramebufferConverter.h
  • platformio.ini
  • scripts/patch_jpegdec.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • platformio.ini
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: build
  • GitHub Check: cppcheck
🧰 Additional context used
🧠 Learnings (4)
📓 Common learnings
Learnt from: martinbrook
Repo: crosspoint-reader/crosspoint-reader PR: 1136
File: scripts/patch_jpegdec.py:54-56
Timestamp: 2026-02-23T19:57:33.257Z
Learning: In the crosspoint-reader project, the patch_jpegdec.py pre-build script uses warn-and-continue (print WARNING and return) rather than fail-fast when patch targets are not found in the JPEGDEC library. This approach is preferred because the patches address edge cases (progressive JPEG AC tables, MCU_SKIP crashes), and a missing target likely indicates either an upstream fix or a library version change requiring manual patch updates. Failing the build would block unrelated development unnecessarily.
Learnt from: martinbrook
Repo: crosspoint-reader/crosspoint-reader PR: 1136
File: scripts/patch_jpegdec.py:20-23
Timestamp: 2026-02-23T19:55:03.054Z
Learning: PlatformIO pre-build scripts in the crosspoint-reader project use the standard SCons pattern `Import("env")` and callback signature `def patch_jpegdec(source, target, env):` without noqa comments or underscore-prefixing unused parameters, following the convention used in other scripts in the project.
📚 Learning: 2026-02-23T19:55:03.054Z
Learnt from: martinbrook
Repo: crosspoint-reader/crosspoint-reader PR: 1136
File: scripts/patch_jpegdec.py:20-23
Timestamp: 2026-02-23T19:55:03.054Z
Learning: PlatformIO pre-build scripts in the crosspoint-reader project use the standard SCons pattern `Import("env")` and callback signature `def patch_jpegdec(source, target, env):` without noqa comments or underscore-prefixing unused parameters, following the convention used in other scripts in the project.

Applied to files:

  • scripts/patch_jpegdec.py
📚 Learning: 2026-02-23T19:57:33.257Z
Learnt from: martinbrook
Repo: crosspoint-reader/crosspoint-reader PR: 1136
File: scripts/patch_jpegdec.py:54-56
Timestamp: 2026-02-23T19:57:33.257Z
Learning: In the crosspoint-reader project, the patch_jpegdec.py pre-build script uses warn-and-continue (print WARNING and return) rather than fail-fast when patch targets are not found in the JPEGDEC library. This approach is preferred because the patches address edge cases (progressive JPEG AC tables, MCU_SKIP crashes), and a missing target likely indicates either an upstream fix or a library version change requiring manual patch updates. Failing the build would block unrelated development unnecessarily.

Applied to files:

  • scripts/patch_jpegdec.py
  • lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp
📚 Learning: 2026-02-23T19:55:03.054Z
Learnt from: martinbrook
Repo: crosspoint-reader/crosspoint-reader PR: 1136
File: scripts/patch_jpegdec.py:20-23
Timestamp: 2026-02-23T19:55:03.054Z
Learning: In the crosspoint-reader project, enable Python linters (e.g., Ruff or Flake8) in CI for all Python scripts. Run linting as part of the CI workflow and fix reported issues to maintain code quality. Do not skip linting for any Python files.

Applied to files:

  • scripts/patch_jpegdec.py
🧬 Code graph analysis (1)
lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp (1)
lib/Epub/Epub/converters/DitherUtils.h (2)
  • applyBayerDither4Level (16-28)
  • drawPixelWithRenderMode (31-40)
🪛 Ruff (0.15.2)
scripts/patch_jpegdec.py

[error] 20-20: Undefined name Import

(F821)


[warning] 23-23: Unused function argument: source

(ARG001)


[warning] 23-23: Unused function argument: target

(ARG001)


[error] 114-114: Undefined name env

(F821)

🔇 Additional comments (12)
lib/Epub/Epub/converters/JpegToFramebufferConverter.h (1)

1-21: Header cleanup aligns with JPEGDEC refactor. Public API remains unchanged.

scripts/patch_jpegdec.py (4)

1-18: Docstring clearly documents patch intent and safety.


34-64: Idempotent AC-table patch logic looks solid.


65-112: MCU_SKIP guard patch is clear and safely idempotent.


114-114: Pre-build hook registration is correct.

lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp (7)

14-50: Per-decode JpegContext is a clean replacement for global state.


76-91: I/O callbacks correctly keep JPEGDEC’s iPos in sync.


98-115: Scale selection logic is straightforward and correct.


117-208: Draw callback is well-structured; bilinear upscaling is a good touch.


212-238: Dimension probe path is safe and well-guarded.


245-288: Decoder setup and validation flow look good.


331-365: Decode + caching completion path is clean and consistent.

@iandchasse
Copy link
Contributor

Maybe these progressive images can be kept as inline and small so that they don't have to scale up on the 480x800 canvas? Maybe that would make them look a bit "better" within our limitations?

@martinbrook
Copy link
Contributor Author

martinbrook commented Feb 23, 2026

Maybe these progressive images can be kept as inline and small so that they don't have to scale up on the 480x800 canvas? Maybe that would make them look a bit "better" within our limitations?

The images do look better if they don't fill the full screen but still we have lost detail.

@martinbrook
Copy link
Contributor Author

martinbrook commented Feb 23, 2026

Maybe these progressive images can be kept as inline and small so that they don't have to scale up on the 480x800 canvas? Maybe that would make them look a bit "better" within our limitations?

Ok I've implemented a scaling factor so instead of filling the space it scales the thumbnail by a set factor. Here are 2x and 3x for the same page I shared in the PR title. It does show quite starkly how much detail we are loosing in the decode phase due to jpegdec "Supports thumbnail (DC-only) decoding of progressive JPEG images"

@martinbrook
Copy link
Contributor Author

martinbrook commented Feb 23, 2026

Here is the JPEGDEC diff

--- ./JPEGDEC/src/jpeg.inl      2026-02-23 23:02:53.387739036 +0000
+++ ./.pio/libdeps/default/JPEGDEC/src/jpeg.inl 2026-02-23 14:47:16.762059668 +0000
@@ -1176,6 +1176,12 @@
             }
         } // if table defined
     }
+    // CrossPoint patch: skip AC tables for progressive JPEG
+    // Progressive JPEG: only DC coefficients are decoded (first scan), so AC
+    // Huffman tables are not needed.  Skip building them to avoid failing on
+    // 11+-bit AC codes that the optimized table builder cannot handle.
+    if (pJPEG->ucMode == 0xc2)
+        return 1;
     // now do AC components (up to 4 tables of 16-bit codes)
     // We split the codes into a short table (9 bits or less) and a long table (first 5 bits are 1)
     for (iTable = 0; iTable < 4; iTable++)
@@ -1851,7 +1857,8 @@
             {
                 //            (*iDCPredictor) |= iPositive;  // in case the scan is run more than once
                 //            pMCU[0] = *iDCPredictor; // store in MCU[0]
-                pMCU[0] |= iPositive;
+                if (iMCU >= 0)
+                    pMCU[0] |= iPositive;
             }
             goto mcu_done; // that's it
         }
@@ -1883,7 +1890,9 @@
             ulCode <<= pJPEG->cApproxBitsLow; // successive approximation shift value
             (*iDCPredictor) += ulCode;
         }
-        pMCU[0] = (short)*iDCPredictor; // store in MCU[0]
+        // CrossPoint patch: guard pMCU write for MCU_SKIP
+        if (iMCU >= 0)
+            pMCU[0] = (short)*iDCPredictor; // store in MCU[0]
     }
     // Now get the other 63 AC coefficients
     pFast = &pJPEG->usHuffAC[pJPEG->ucACTable * HUFF11SIZE];
@@ -3766,7 +3775,6 @@
 #endif // ESP32S3_SIMD

 #ifdef HAS_NEON
-    if (x+8 <= iPitch && (iPitch & 15) == 0) { // only for non-clipped MCUs
     if (pJPEG->ucPixelType == RGB8888) {
        int8x8_t i88Cr, i88Cb;
        uint8x16_t u816YL, u816YR;
@@ -3999,7 +4007,6 @@
           } // for each row
       return;
       } // 16bpp
-      } // not clipped
 #endif // HAS_NEON

 #ifdef HAS_SSE

@martinbrook
Copy link
Contributor Author

martinbrook commented Feb 24, 2026

I noticed there might be some code in upstream which might help here. Enable JPEGMakeHuffTables_Slow — remove the #ifdef FUTURE wrapper but its still not going to help us do a full decode due to limited ram. So this is as far as we can go with progressive I think ie we can decode but the quality is sub-optimal. If a user needs improved quality then the progressive jpegs need pre processing using the caliber plugin or webtool. Another option would be to add automatic conversion by the browser server side in image upload to the device.

As a comparison of the quality difference with a pre processed image here is

on device processed then pre processed file (using https://baseliner.radtke96.workers.dev/)

@martinbrook
Copy link
Contributor Author

martinbrook commented Feb 24, 2026

Will work on getting timings to compare picojpeg with jpegdec so this can be considered and I'll leave the x2, x3 scaling changes out for now

@martinbrook
Copy link
Contributor Author

martinbrook commented Feb 24, 2026

Good performance increase from JPEGDEC over picojpeg cc @bitbank2 thanks

Baseline JPEG Decode Performance: JPEGDEC vs picojpeg

Tested with test_jpeg_images.epub on device (ESP32-C3), first decode (no cache).

Image Source Output picojpeg JPEGDEC Speedup
jpeg_format.jpg 350x250 350x250 313 ms 256 ms 1.2x
grayscale_test.jpg 400x600 400x600 768 ms 661 ms 1.2x
gradient_test.jpg 400x500 400x500 707 ms 597 ms 1.2x
centering_test.jpg 350x400 350x400 502 ms 412 ms 1.2x
scaling_test.jpg 1200x1500 464x580 5487 ms 1114 ms 4.9x
wide_scaling_test.jpg 1807x736 464x188 4237 ms 642 ms 6.6x
cache_test_1.jpg 400x300 400x300 422 ms 348 ms 1.2x
cache_test_2.jpg 400x300 400x300 424 ms 349 ms 1.2x

Summary

  • 1:1 scale: ~20% faster consistently across all image sizes
  • Downscaled images: 5-7x faster — JPEGDEC's built-in coarse scaling (1/2, 1/4, 1/8) skips DCT coefficient decoding for pixels that would be discarded. The 1200x1500 image dropped from 5.5s to 1.1s.
  • Page render time for the large scaling_test chapter dropped from 7.9s to 3.6s total
  • Potential to speed up further in the callback by using fixed point
  • Potential to speed up the bilinear interpolation by reducing the boundry checking bu being "strategic" about it. From @bitbank2

@bitbank2
Copy link

yeah, makes sense. cachingpaging all that data to sd card would probably be slow as heck, too

There was one other suggestion but again needs discussion with upstream. 1. Change JPEGDEC's progressive support — patch the Huffman table builder to handle 11+ bit AC codes, so it can fully decode progressive JPEGs instead of DC-only. This is the ideal fix but requires deep library modification.

There's no benefit to patch the Huffman table builder. The problem with progressive JPEGs is that you need to have even MORE RAM than the fully decoded image. The image is decoded in 'scans' where each scan provides some info for the MCU coefficients. I provided a functional workaround by decoding just the first scan (which is usually only the DC values). This allows for decoding a 1/8th scale image on constrained microcontrollers. Allowing decode of full sized progressive images would only be practical on something with lots of RAM (e.g. ESP32-S3 w/PSRAM).

@martinbrook
Copy link
Contributor Author

martinbrook commented Feb 25, 2026

@bitbank2 much improved numbers after optimising (see below for details)
@daveallie ready for review

Baseline JPEG Decode Performance: picojpeg vs JPEGDEC (float) vs JPEGDEC (fixed-point)

Tested with test_jpeg_images.epub on device (ESP32-C3), first decode (no cache).

Image Source Output picojpeg JPEGDEC float JPEGDEC fixed-point vs picojpeg vs float
jpeg_format.jpg 350x250 350x250 313 ms 256 ms 104 ms 3.0x 2.5x
grayscale_test.jpg 400x600 400x600 768 ms 661 ms 246 ms 3.1x 2.7x
gradient_test.jpg 400x500 400x500 707 ms 597 ms 247 ms 2.9x 2.4x
centering_test.jpg 350x400 350x400 502 ms 412 ms 169 ms 3.0x 2.4x
scaling_test.jpg 1200x1500 464x580 5487 ms 1114 ms 668 ms 8.2x 1.7x
wide_scaling_test.jpg 1807x736 464x188 4237 ms 642 ms 497 ms 8.5x 1.3x
cache_test_1.jpg 400x300 400x300 422 ms 348 ms 141 ms 3.0x 2.5x
cache_test_2.jpg 400x300 400x300 424 ms 349 ms 142 ms 3.0x 2.5x

Summary

  • 1:1 scale (fixed-point vs float): ~2.5x faster — eliminating software float on the FPU-less ESP32-C3 is the dominant win
  • 1:1 scale (fixed-point vs picojpeg): ~3.0x faster overall
  • Downscaled images (vs picojpeg): 8-9x faster — JPEGDEC's coarse scaling + fixed-point draw callback
  • Downscaled images (fixed-point vs float): 1.3-1.7x — less dramatic since JPEG library decode dominates over the draw callback for fewer output pixels
  • The fixed-point optimization alone (vs float JPEGDEC) saved ~60% of render time on 1:1 images, confirming that software float emulation was the primary bottleneck in the draw callback

Changes made

  • Fixed-point 16.16 arithmetic — Replaced all float operations in the draw callback with integer math. On ESP32-C3 (no FPU), each float multiply was ~10-15 cycles vs 1 cycle for integer. The bilinear interpolation now uses the form (p0 * (FP_ONE - f) + p1 * f) >> FP_SHIFT which also fixes the unsigned subtraction bug (p10 - p00 wrapping for uint8_t).

-Three separate paths:

  • 1:1 fast path (line 170) — When fineScaleFP == FP_ONE, zero scaling math. Direct pixel lookup with offset arithmetic only.

  • Bilinear upscale (line 194) — For progressive JPEG DC-only decode. Uses strategic boundary splitting:

    • Pre-computes safeXStart/safeXEnd where source X neighbors are guaranteed in-bounds
    • Left edge loop (0-2 pixels): full boundary clamping
    • Interior loop (bulk of pixels): no boundary checks at all
    • Right edge loop (1-8 pixels): boundary clamping
  • Nearest-neighbor downscale (line 297) — Simple fixed-point coordinate mapping with minimal clamping.

  • Pre-clamped screen bounds (line 156) — Destination ranges are clamped against screen dimensions once before any loops, eliminating 4 per-pixel branch comparisons (outX < 0, outX >=screenWidth, outY < 0, outY >= screenHeight).

Replace all float operations with 16.16 fixed-point integer math in
jpegDrawCallback. The ESP32-C3 has no FPU, so each float multiply was
~10-15 cycles vs 1 cycle for integer — this alone yields ~2.5x speedup
on 1:1 scale images.

Key changes:
- Fixed-point 16.16 for all scaling/interpolation in the hot path
- 1:1 fast path that skips all scaling math entirely
- Pre-clamp destination ranges to screen bounds (eliminates 4 per-pixel
  branch comparisons)
- Strategic boundary splitting for bilinear upscale: branchless interior
  loop with clamping only at left/right edge pixels
- Fix unsigned subtraction bug in bilinear (uint8_t wrap on p10 - p00)
  by using weighted sum form: (p0 * (1-f) + p1 * f) >> 16
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR switches the EPUB JPEG image decoding path from picojpeg to bitbank2/JPEGDEC, enabling faster decoding (including coarse scaling), 8-bit grayscale output, and block-based streaming decode via callbacks. It also introduces a PlatformIO pre-build patch script to apply two upstream JPEGDEC fixes needed for progressive JPEG handling and EIGHT_BIT_GRAYSCALE stability.

Changes:

  • Add a PlatformIO pre-build script to patch JPEGDEC’s jpeg.inl for progressive JPEG and MCU_SKIP handling.
  • Add JPEGDEC as a dependency and wire the patch script into the PlatformIO build.
  • Rewrite JpegToFramebufferConverter to use JPEGDEC callbacks with fixed-point scaling and optional bilinear upscaling for progressive DC-only output.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
scripts/patch_jpegdec.py Adds an idempotent build-time patcher for two JPEGDEC issues impacting progressive and grayscale decode.
platformio.ini Registers the pre-build patch script and adds bitbank2/JPEGDEC to lib_deps.
lib/Epub/Epub/converters/JpegToFramebufferConverter.h Removes the old picojpeg read callback declaration (no longer needed).
lib/Epub/Epub/converters/JpegToFramebufferConverter.cpp Replaces picojpeg decode loop with JPEGDEC open/decode + draw callback rendering and fixed-point scaling.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@martinbrook
Copy link
Contributor Author

Thanks for the review @osteotek , there are still some optimisations here to be had but I'd rather do those in a followup as they are common to both png and jpeg decode paths.

@osteotek
Copy link
Member

So what's the conclusion on upstreaming the patch?

osteotek
osteotek previously approved these changes Feb 26, 2026
@martinbrook
Copy link
Contributor Author

martinbrook commented Feb 26, 2026

So what's the conclusion on upstreaming the patch?

I've raised an issue upstream

@martinbrook martinbrook marked this pull request as draft February 26, 2026 22:53
@martinbrook
Copy link
Contributor Author

Found an issue when this is built clean so not ready yet

AddPreAction("buildprog", ...) deferred patching until the link step,
after JPEGDEC was already compiled from unpatched source. Call the
patch function immediately when the pre: script runs so the source is
patched before compilation begins.
@martinbrook martinbrook marked this pull request as ready for review February 27, 2026 00:17
@martinbrook
Copy link
Contributor Author

RFR Again

osteotek
osteotek previously approved these changes Feb 27, 2026
@daveallie
Copy link
Member

Huge, thanks @martinbrook!

@daveallie daveallie merged commit 2b25f4d into crosspoint-reader:master Mar 1, 2026
6 checks passed
laird pushed a commit to laird/crosspoint-claw that referenced this pull request Mar 1, 2026
…nt-reader#1136)

## Summary

Replaces the picojpeg library with bitbank2/JPEGDEC for JPEG decoding in
the EPUB image pipeline. JPEGDEC provides built-in coarse scaling (1/2,
1/4, 1/8), 8-bit grayscale output, and streaming block-based decoding
via callbacks.

Includes a pre-build patch script for two JPEGDEC changes affecting
progressive JPEG support and EIGHT_BIT_GRAYSCALE mode.

Closes crosspoint-reader#912 

## Additional Context
# Example progressive jpeg 

<img
src="https://github.com/user-attachments/assets/e63bb4f8-f862-4aa0-a01f-d1ef43a4b27a"
width="400" height="800" />

Good performance increase from JPEGDEC over picojpeg cc @bitbank2 thanks

## Baseline JPEG Decode Performance: picojpeg vs JPEGDEC (float in
callback) vs JPEGDEC (fixed-point in callback)

Tested with `test_jpeg_images.epub` on device (ESP32-C3), first decode
(no cache).

| Image | Source | Output | picojpeg | JPEGDEC float | JPEGDEC
fixed-point | vs picojpeg | vs float |

|-------|--------|--------|----------|---------------|---------------------|-------------|----------|
| jpeg_format.jpg | 350x250 | 350x250 | 313 ms | 256 ms | **104 ms** |
**3.0x** | **2.5x** |
| grayscale_test.jpg | 400x600 | 400x600 | 768 ms | 661 ms | **246 ms**
| **3.1x** | **2.7x** |
| gradient_test.jpg | 400x500 | 400x500 | 707 ms | 597 ms | **247 ms** |
**2.9x** | **2.4x** |
| centering_test.jpg | 350x400 | 350x400 | 502 ms | 412 ms | **169 ms**
| **3.0x** | **2.4x** |
| scaling_test.jpg | 1200x1500 | 464x580 | 5487 ms | 1114 ms | **668
ms** | **8.2x** | **1.7x** |
| wide_scaling_test.jpg | 1807x736 | 464x188 | 4237 ms | 642 ms | **497
ms** | **8.5x** | **1.3x** |
| cache_test_1.jpg | 400x300 | 400x300 | 422 ms | 348 ms | **141 ms** |
**3.0x** | **2.5x** |
| cache_test_2.jpg | 400x300 | 400x300 | 424 ms | 349 ms | **142 ms** |
**3.0x** | **2.5x** |

  ### Summary

- **1:1 scale (fixed-point vs float)**: ~2.5x faster — eliminating
software float on the FPU-less ESP32-C3 is the dominant win
- **1:1 scale (fixed-point vs picojpeg)**: ~3.0x faster overall
- **Downscaled images (vs picojpeg)**: 8-9x faster — JPEGDEC's coarse
scaling + fixed-point draw callback
- **Downscaled images (fixed-point vs float)**: 1.3-1.7x — less dramatic
since JPEG library decode dominates over the draw callback for fewer
output pixels
- The fixed-point optimization alone (vs float JPEGDEC) saved **~60% of
render time** on 1:1 images, confirming that software float emulation
was the primary bottleneck in the draw callback
- See thread for discussions on quality of progressive images,
crosspoint-reader#1136 (comment)
- and the conclusion
crosspoint-reader#1136 (comment)
- Proposal to improve quality added at
crosspoint-reader#1179
---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**< PARTIALLY >**_

---------

Co-authored-by: Dave Allie <[email protected]>
laird added a commit to laird/crosspoint-claw that referenced this pull request Mar 1, 2026
Cherry-picked upstream fixes and features applied via rescue-1.0.0 worktree:
- fix: properly implement requestUpdateAndWait() (crosspoint-reader#1218)
- fix: Hide unusable button hints in empty directory (crosspoint-reader#1253)
- fix: add missing keyboard metrics to Lyra3CoversTheme (crosspoint-reader#1101)
- fix: remove bookProgressBarHeight from Lyra3CoversTheme
- feat: replace picojpeg with JPEGDEC for JPEG decoding (crosspoint-reader#1136)
- feat: WIFI pill, feed log fix, JPEGDEC version string
- feat: Add git branch to version string (crosspoint-reader#1225)
- fix: navigate directly to QR code after DZ auto-connect
- perf: Removed unused ConfirmationActivity member (crosspoint-reader#1234)
- refactor: Simplify new setting introduction (crosspoint-reader#1086)
- fix: UI fonts, settings stack overflow, PULSR theme name
- fix: convert SettingsList to push_back (prevent stack overflow)

All commits built and verified on device (confirmed 1.1.0-dev+d1e786a).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replace use of picojpeg with jpegdec

8 participants