Skip to content

perf: Reduce overall flash usage by 30.7% by compressing built-in fonts#831

Merged
daveallie merged 13 commits intocrosspoint-reader:masterfrom
adriancaruana:font-compression
Feb 19, 2026
Merged

perf: Reduce overall flash usage by 30.7% by compressing built-in fonts#831
daveallie merged 13 commits intocrosspoint-reader:masterfrom
adriancaruana:font-compression

Conversation

@adriancaruana
Copy link
Contributor

@adriancaruana adriancaruana commented Feb 11, 2026

Summary

What is the goal of this PR?

Compress reader font bitmaps to reduce flash usage by 30.7%.

What changes are included?

  • New EpdFontGroup struct and extended EpdFontData with groups/groupCount fields
  • --compress flag in fontconvert.py: groups glyphs (ASCII base group + groups of 8) and compresses each with raw DEFLATE
  • FontDecompressor class with 4-slot LRU cache for on-demand decompression during rendering
  • GfxRenderer transparently routes bitmap access through getGlyphBitmap() (compressed or direct flash)
  • Uses uzlib for decompression with minimal heap overhead.
  • 48 reader fonts (Bookerly, NotoSans 12-18pt, OpenDyslexic) regenerated with compression; 5 UI fonts unchanged
  • Round-trip verification script (verify_compression.py) runs as part of font generation

Additional Context

Flash & RAM

baseline font-compression Difference
Flash (ELF) 6,302,476 B (96.2%) 4,365,022 B (66.6%) -1,937,454 B (-30.7%)
firmware.bin 6,468,192 B 4,531,008 B -1,937,184 B (-29.9%)
RAM 101,700 B (31.0%) 103,076 B (31.5%) +1,376 B (+0.5%)

Script-Based Grouping (Cold Cache)

Comparison of uncompressed baseline vs script-based group compression (4-slot LRU cache, cleared each page). Glyphs are grouped by Unicode block (ASCII, Latin-1, Latin Extended-A, Combining Marks, Cyrillic, General Punctuation, etc.) instead of sequential groups of 8.

Render Time

Baseline Compressed (cold cache) Difference
Median 414.9 ms 431.6 ms +16.7 ms (+4.0%)
Pages 37 37

Memory Usage

Baseline Compressed (cold cache) Difference
Heap free (median) 187.0 KB 176.3 KB -10.7 KB
Heap free (min) 186.0 KB 166.5 KB -19.5 KB
Largest block (median) 148.0 KB 128.0 KB -20.0 KB
Largest block (min) 148.0 KB 120.0 KB -28.0 KB

Cache Effectiveness

Misses/page Hit rate
Compressed (cold cache) 2.1 99.85%

AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? YES
Implementation was done by Claude Code (Opus 4.6) based on a plan developed collaboratively. All generated font headers were verified with an automated round-trip decompression test. The firmware was compiled successfully but has not yet been tested on-device.

@adriancaruana adriancaruana changed the title Reduce flash usage by compressing built-in fonts. perf: Reduce flash usage by compressing built-in fonts Feb 11, 2026
@adriancaruana adriancaruana changed the title perf: Reduce flash usage by compressing built-in fonts perf: Reduce overall flash usage by 24% by compressing built-in fonts Feb 11, 2026
@daveallie
Copy link
Member

This is obviously a slightly different approach to my per-glyph approach (#254), but I'm curious to know about on-device render performance vs master. In my branch, it basically added another 50% to the render time for a full page of text (moving from 700ms -> 1000ms)

@ngxson
Copy link
Contributor

ngxson commented Feb 11, 2026

  • Runtime heap cost is ~19 KB (cache buffers + shared decompressor). The tinfl_decompressor allocation (~11 KB) is shared with ZipFile, so it's not a net-new cost.

I'm quite curious how you measured these numbers without testing it on-device though. If I understand correctly, the heap tracking function from your emulator doesn't return the real value. Also, just note that because the emulated firmware is compiled to host arch (ARM64 on mac), it will behave differently in terms of memory usage. Remind that the ESP32C3 uses RISC-V 32-bit.

Btw, I'm interested in making an emulator too. A bit off-topic, the direction of your emulator is good, but I think it will be even cleaner if it can:

  • Separate stub layers: arduino / crosspoint-specific logic HAL
  • Use upstream freeRTOS with proper Portable layer
  • Maybe make part of the emulator as a custom platform on platformio?

@adriancaruana
Copy link
Contributor Author

adriancaruana commented Feb 11, 2026

@daveallie I reckon if you'd followed through with your idea of adding a cache, the PR would have been much closer to being merged. However, the grouping approach significantly increases the compression ratio, and, in relative terms when compiled and run on my laptop, modestly improves the glyph retrieval time per page. Note I also tested LZ4, which is much faster than miniz in decompression, however the cache seems to hide the decompression speed differences. The benchmark was run on an entire English EPUB. These obviously do not replace testing on-device, which I unfortunately cannot do yet since I'm still waiting for my device to be shipped. 😢

Approach Cache Hit Rate Time/page Flash size
LZ4 group yes 99.99% 0.02 ms 75.6% of original
DEFLATE group yes 99.99% 0.05 ms 53.4% of original
LZ4 per-glyph yes 84.57% 0.07 ms 89.0% of original
DEFLATE per-glyph yes 84.57% 0.12 ms 81.7% of original
LZ4 per-glyph no 0.28 ms 89.0% of original
DEFLATE per-glyph no 7.54 ms 81.7% of original
LZ4 group no 10.14 ms 75.6% of original
DEFLATE group no 103.18 ms 53.4% of original

@ngxson Claude seems to be able to infer what the heap allocations would be just by reading the code. I'd love to validate this -- maybe when my device arrives!

The ~11 KB tinfl_decompressor figure is accurate. The struct contains 3 Huffman tables at ~3,488 bytes each plus overhead, totaling ~11,090 bytes.
The ~8 KB cache buffer figure is a reasonable midpoint but the actual range is wide. The 4-slot LRU cache holds decompressed group buffers whose sizes vary significantly by font and point size: the low-end is ~4 KB for 8pt, and the high-end is 20 KB for 18 pt bold italic.

One misleading comment in the original PR is that the ~11 KB allocation for tinfl_decompressor is not a net new cost: this is wrong, it's permanently allocated now (because it's used frequently).

@ngxson
Copy link
Contributor

ngxson commented Feb 11, 2026

Claude seems to be able to infer what the heap allocations would be just by reading the code.

Just note that I have seen instances where Claude straight out fake the results to make it look more appealing. If any test results (and even test scripts) are made by AI, please (1) take it as a grain of salt and (2) explicitly state that the result is generated by AI

@adriancaruana
Copy link
Contributor Author

I have seen instances where Claude straight out fake the results to make it look more appealing

I've seen this too. It would be great if you can test this build out and see what the heap allocations are actually like. Obviously this PR should not be merged until someone does this.

@ngxson
Copy link
Contributor

ngxson commented Feb 12, 2026

I've seen this too. It would be great if you can test this build out and see what the heap allocations are actually like. Obviously this PR should not be merged until someone does this.

I am sorry but my job here is to review the code and optionally do analysis if things don't look right.

The author of each PR should be the first person to verify the results before pushing the PR. Especially important when the author gives any numbers in the PR description:

  • Runtime heap cost is ~19 KB (cache buffers + shared decompressor). The tinfl_decompressor allocation (~11 KB) is shared with ZipFile, so it's not a net-new cost.
  • All 48 compressed fonts pass round-trip verification (decompress every group, check sizes and glyph offsets).

Compress reader font bitmaps to reduce flash usage by 1.49 MB (A saving of 23.9%).

@adriancaruana
Copy link
Contributor Author

adriancaruana commented Feb 13, 2026

Compress reader font bitmaps to reduce flash usage by 1.49 MB (A saving of 23.9%).

@ngxson I validated this claim myself, and it should be self-evident from the PR diff (100k line deletions * 16 bytes per line = ~1.6 MB). Though you're right to be skeptical of the other claims which are not so self-evident (as was I). So, I snooped around in your repos and was able to run your hardware emulator to benchmark the results on this dev board which Amazon next-day delivered (thanks for the idea, neat!). (PS: I can show you how to fix the IO issues you mentioned in the README if you like). Note: the benchmarks below contain no sd card IO, so they should be representative of the actual xteink performance.

image

@daveallie The page render time was basically unaffected by the font compression. The key is to minimise the number of decompressions needed to render a page. This is achieved with glyph-grouping (which is more compression-efficient anyway) and with caching (i.e., don't decompress the same glyph twice within each page and/or across pages). The obvious downside is heap allocation. The heap size cost for Bookerly 14pt is ~19 KB. This can be freed after each page render, however it means that the next page starts with a cold cache (though this is no big deal, so potentially this should be removed before merging).

Benchmark summary (Written by Claude from benchmark stats)

Font Compression Benchmark

On-device benchmark of group-based DEFLATE font compression on ESP32-C3 (RISC-V), run via the emulated environment (pio run -e emulated). The test book was rendered using the default font (Bookerly 14pt, regular weight) across multiple chapters. Pages with fewer than 800 glyphs (short end-of-chapter pages) are excluded.

Configurations

Config Description
Baseline Uncompressed fonts — direct bitmap pointer access
Cached DEFLATE group compression with 4-slot LRU cache (normal operation)
Cold cache Same as cached, but cache cleared at each p\age boundary — forces 1–3 decompressions per page start
No cache Cache fully disabled — every glyph triggers a full group decompression

Render Time

Baseline Cached Cold cache No cache
Median 414.9 ms 414.2 ms (-0.2%) 422.0 ms (+1.7%) 3,090 ms (+645%)
Mean 421.5 ms 422.6 ms (+0.3%) 431.3 ms (+2.3%) 3,084 ms (+632%)
P95 511.9 ms 506.1 ms (-1.1%) 521.3 ms (+1.8%) 3,294 ms (+543%)
Pages 37 47 35 27

With the cache enabled, compression adds no measurable overhead. Even starting cold on every page only costs +1.7%.

Memory Usage

Baseline Cached Cold cache No cache
Heap free (median) 168.3 KB 149.4 KB (-11%) 149.6 KB (-11%) 129.1 KB (-23%)
Heap free (min) 166.7 KB 140.7 KB 140.2 KB 127.1 KB
Largest block (median) 148.0 KB 128.0 KB 124.0 KB 61.0 KB
Largest block (min) 148.0 KB 122.0 KB 68.0 KB 43.0 KB

The cached configuration uses ~19 KB more heap than baseline, from the decompressed group buffer (~7 KB) and the tinfl_decompressor struct (~11 KB). The no-cache configuration fragments heap significantly — the largest free block drops from 148 KB to 61 KB median (43 KB worst case) due to repeated malloc/decompress/free cycles.

Cache Effectiveness

Hits Misses Hit rate
Cached 67,370 4 99.99%
Cold cache 50,629 83 99.84%
No cache 0 38,814 0%

English text fits almost entirely in one font group. The 4-slot LRU cache holds it for the entire page, yielding a near-perfect hit rate with only 1–3 decompressions needed per page.

Benchmark data:
baseline.csv
compressed_cold_cache.csv
compressed_with_cache.csv
compressed_without_cache.csv

@adriancaruana
Copy link
Contributor Author

I've swaped out miniz with uzlib, which essentially reduces the heap allocation of the decompressor to 0. The remaining heap allocation should just be for the LRU cache, which is now being freed between pages (because paying the cost of group decompression per-page is fine).

Render Time

Baseline miniz cold uzlib cold uzlib vs miniz
Median 414.9 ms 422.0 ms 428.0 ms +6.0 ms (+1.4%)
Mean 421.5 ms 431.3 ms 432.7 ms +1.4 ms (+0.3%)
P95 511.9 ms 521.3 ms 523.4 ms +2.1 ms (+0.4%)
Pages 37 35 38

Render time is equivalent — the +1.4% median difference is within run-to-run variance.

Memory Usage

Baseline miniz cold uzlib cold uzlib vs miniz
Heap free (median) 168.3 KB 149.6 KB 159.6 KB +10.0 KB (+6.7%)
Heap free (min) 166.7 KB 140.2 KB 151.4 KB +11.2 KB (+8.0%)
Largest block (median) 148.0 KB 124.0 KB 136.0 KB +12.0 KB (+9.7%)
Largest block (min) 148.0 KB 68.0 KB 126.0 KB +58.0 KB (+85.3%)

uzlib reclaims ~10–12 KB of heap, consistent with eliminating the ~11 KB tinfl_decompressor struct. The largest-block-min improvement (68 KB → 126 KB) indicates miniz was causing significant heap fragmentation in worst-case scenarios.

compressed_cold_cache_uzlib.csv

@ngxson
Copy link
Contributor

ngxson commented Feb 13, 2026

100k line deletions * 16 bytes per line = ~1.6 MB

And on the PR description:

Compress reader font bitmaps to reduce flash usage by 1.49 MB (A saving of 23.9%).

You already given 2 different numbers.

Sorry but I will stop commenting and reviewing this PR because there are too many inconsistencies in term of report results. Will leave this for other maintainers to review.

@adriancaruana adriancaruana changed the title perf: Reduce overall flash usage by 24% by compressing built-in fonts perf: Reduce overall flash usage by 30.7% by compressing built-in fonts Feb 13, 2026
@adriancaruana
Copy link
Contributor Author

Okay, I'll leave this here. See here for a detailed breakdown of differences in flash size, render time, and heap allocation.

If the maintainers would prefer, I can close this PR and re-open so the history is clearer.

Copy link
Member

@daveallie daveallie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall the changes look fine and in local testing, the performance looks like ~2.5%+ increase to rendering time. A very valid trade-off

@daveallie
Copy link
Member

@coderabbitai review

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 16, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 16, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds grouped DEFLATE compression for built-in fonts, a new FontDecompressor with a 4‑slot LRU cache and uzlib-based inflate, tools to emit and verify compressed font headers, renderer integration for decompression, and cache-clearing calls at page-render entry points.

Changes

Cohort / File(s) Summary
Font decompression core
lib/EpdFont/FontDecompressor.h, lib/EpdFont/FontDecompressor.cpp, lib/EpdFont/EpdFontData.h
Adds FontDecompressor class (init/deinit, 4-slot LRU cache, group-based decompression, getBitmap) and EpdFontGroup support in EpdFontData.
Font conversion & verification
lib/EpdFont/scripts/fontconvert.py, lib/EpdFont/scripts/convert-builtin-fonts.sh, lib/EpdFont/scripts/verify_compression.py
fontconvert.py gains --compress to emit grouped raw-DEFLATE compressed glyphs and group metadata; conversion script invokes compression and runs verify_compression.py; new verifier performs round-trip decompression and glyph bounds checks.
Renderer integration
lib/GfxRenderer/GfxRenderer.h, lib/GfxRenderer/GfxRenderer.cpp
Renderer gains FontDecompressor* member, setFontDecompressor() and clearFontCache() APIs, a getGlyphBitmap() helper, and uses decompressor for group-compressed fonts.
Builtin font initializers
lib/EpdFont/builtinFonts/notosans_8_regular.h, lib/EpdFont/builtinFonts/ubuntu_10_*.h, lib/EpdFont/builtinFonts/ubuntu_12_*.h
Updated EpdFontData initializers to supply nullptr, 0 for the new groups pointer/count fields produced by the converter.
uzlib library (decompression)
lib/uzlib/src/tinflate.c, lib/uzlib/src/uzlib.h, lib/uzlib/src/defl_static.h, lib/uzlib/src/tinf.h, lib/uzlib/src/tinf_compat.h, lib/uzlib/src/uzlib_conf.h, lib/uzlib/library.json
Adds uzlib inflate implementation and public API, deflate helper prototypes, compatibility headers, config header, and library manifest to support raw DEFLATE decompression and checksums.
Application wiring
src/main.cpp, src/activities/reader/EpubReaderActivity.cpp, src/activities/reader/TxtReaderActivity.cpp
Introduces a global FontDecompressor, initializes and registers it with the renderer, and calls renderer.clearFontCache() at page-render entry points.
Build/auxiliary changes
bin/clang-format-fix, lib/ZipFile/ZipFile.cpp
clang-format-fix excludes lib/uzlib/; ZipFile.cpp tightens inflator lifecycle and checks, making the inflater static/internal.
Other generated headers
lib/EpdFont/builtinFonts/*
Generated headers may now include Groups arrays and compressed Bitmaps when created with the --compress option.

Sequence Diagram

sequenceDiagram
    participant Renderer as GfxRenderer
    participant Decompressor as FontDecompressor
    participant uzlib as uzlib
    participant FontData as EpdFontData

    Renderer->>Renderer: getGlyphBitmap(fontData, glyph)
    alt fontData has Groups (compressed)
        Renderer->>Decompressor: getGroupIndex(fontData, glyphIndex)
        Decompressor-->>Renderer: groupIndex
        Renderer->>Decompressor: findInCache(fontData, groupIndex)
        alt cache hit
            Decompressor-->>Renderer: pointer to decompressed buffer
        else cache miss
            Decompressor->>Decompressor: findEvictionCandidate()
            Decompressor-->>Renderer: slot
            Decompressor->>FontData: read compressed group bytes
            Decompressor->>uzlib: uzlib_uncompress(compressed → dest)
            uzlib-->>Decompressor: decompressed bytes
            Decompressor->>Decompressor: store in cache slot (valid, lastUsed)
            Decompressor-->>Renderer: pointer to decompressed buffer
        end
        Renderer->>Renderer: compute glyph ptr from group offset and return bitmap
    else no groups (uncompressed)
        Renderer-->>Renderer: return bitmap from FontData->Bitmaps at glyph->dataOffset
    end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Merge Conflict Detection ⚠️ Warning ❌ Merge conflicts detected (158 files):

⚔️ .gitignore (content)
⚔️ README.md (content)
⚔️ SCOPE.md (content)
⚔️ bin/clang-format-fix (content)
⚔️ docs/hyphenation-trie-format.md (content)
⚔️ docs/images/wifi/webserver_files.png (content)
⚔️ docs/images/wifi/webserver_upload.png (content)
⚔️ docs/webserver.md (content)
⚔️ lib/EpdFont/EpdFontData.h (content)
⚔️ lib/EpdFont/builtinFonts/bookerly_12_bold.h (content)
⚔️ lib/EpdFont/builtinFonts/bookerly_12_bolditalic.h (content)
⚔️ lib/EpdFont/builtinFonts/bookerly_12_italic.h (content)
⚔️ lib/EpdFont/builtinFonts/bookerly_12_regular.h (content)
⚔️ lib/EpdFont/builtinFonts/bookerly_14_bold.h (content)
⚔️ lib/EpdFont/builtinFonts/bookerly_14_bolditalic.h (content)
⚔️ lib/EpdFont/builtinFonts/bookerly_14_italic.h (content)
⚔️ lib/EpdFont/builtinFonts/bookerly_14_regular.h (content)
⚔️ lib/EpdFont/builtinFonts/bookerly_16_bold.h (content)
⚔️ lib/EpdFont/builtinFonts/bookerly_16_bolditalic.h (content)
⚔️ lib/EpdFont/builtinFonts/bookerly_16_italic.h (content)
⚔️ lib/EpdFont/builtinFonts/bookerly_16_regular.h (content)
⚔️ lib/EpdFont/builtinFonts/bookerly_18_bold.h (content)
⚔️ lib/EpdFont/builtinFonts/bookerly_18_bolditalic.h (content)
⚔️ lib/EpdFont/builtinFonts/bookerly_18_italic.h (content)
⚔️ lib/EpdFont/builtinFonts/bookerly_18_regular.h (content)
⚔️ lib/EpdFont/builtinFonts/notosans_12_bold.h (content)
⚔️ lib/EpdFont/builtinFonts/notosans_12_bolditalic.h (content)
⚔️ lib/EpdFont/builtinFonts/notosans_12_italic.h (content)
⚔️ lib/EpdFont/builtinFonts/notosans_12_regular.h (content)
⚔️ lib/EpdFont/builtinFonts/notosans_14_bold.h (content)
⚔️ lib/EpdFont/builtinFonts/notosans_14_bolditalic.h (content)
⚔️ lib/EpdFont/builtinFonts/notosans_14_italic.h (content)
⚔️ lib/EpdFont/builtinFonts/notosans_14_regular.h (content)
⚔️ lib/EpdFont/builtinFonts/notosans_16_bold.h (content)
⚔️ lib/EpdFont/builtinFonts/notosans_16_bolditalic.h (content)
⚔️ lib/EpdFont/builtinFonts/notosans_16_italic.h (content)
⚔️ lib/EpdFont/builtinFonts/notosans_16_regular.h (content)
⚔️ lib/EpdFont/builtinFonts/notosans_18_bold.h (content)
⚔️ lib/EpdFont/builtinFonts/notosans_18_bolditalic.h (content)
⚔️ lib/EpdFont/builtinFonts/notosans_18_italic.h (content)
⚔️ lib/EpdFont/builtinFonts/notosans_18_regular.h (content)
⚔️ lib/EpdFont/builtinFonts/notosans_8_regular.h (content)
⚔️ lib/EpdFont/builtinFonts/opendyslexic_10_bold.h (content)
⚔️ lib/EpdFont/builtinFonts/opendyslexic_10_bolditalic.h (content)
⚔️ lib/EpdFont/builtinFonts/opendyslexic_10_italic.h (content)
⚔️ lib/EpdFont/builtinFonts/opendyslexic_10_regular.h (content)
⚔️ lib/EpdFont/builtinFonts/opendyslexic_12_bold.h (content)
⚔️ lib/EpdFont/builtinFonts/opendyslexic_12_bolditalic.h (content)
⚔️ lib/EpdFont/builtinFonts/opendyslexic_12_italic.h (content)
⚔️ lib/EpdFont/builtinFonts/opendyslexic_12_regular.h (content)
⚔️ lib/EpdFont/builtinFonts/opendyslexic_14_bold.h (content)
⚔️ lib/EpdFont/builtinFonts/opendyslexic_14_bolditalic.h (content)
⚔️ lib/EpdFont/builtinFonts/opendyslexic_14_italic.h (content)
⚔️ lib/EpdFont/builtinFonts/opendyslexic_14_regular.h (content)
⚔️ lib/EpdFont/builtinFonts/opendyslexic_8_bold.h (content)
⚔️ lib/EpdFont/builtinFonts/opendyslexic_8_bolditalic.h (content)
⚔️ lib/EpdFont/builtinFonts/opendyslexic_8_italic.h (content)
⚔️ lib/EpdFont/builtinFonts/opendyslexic_8_regular.h (content)
⚔️ lib/EpdFont/builtinFonts/ubuntu_10_bold.h (content)
⚔️ lib/EpdFont/builtinFonts/ubuntu_10_regular.h (content)
⚔️ lib/EpdFont/builtinFonts/ubuntu_12_bold.h (content)
⚔️ lib/EpdFont/builtinFonts/ubuntu_12_regular.h (content)
⚔️ lib/EpdFont/scripts/convert-builtin-fonts.sh (content)
⚔️ lib/EpdFont/scripts/fontconvert.py (content)
⚔️ lib/Epub/Epub.cpp (content)
⚔️ lib/Epub/Epub.h (content)
⚔️ lib/Epub/Epub/Page.cpp (content)
⚔️ lib/Epub/Epub/Page.h (content)
⚔️ lib/Epub/Epub/Section.cpp (content)
⚔️ lib/Epub/Epub/blocks/Block.h (content)
⚔️ lib/Epub/Epub/blocks/TextBlock.cpp (content)
⚔️ lib/Epub/Epub/blocks/TextBlock.h (content)
⚔️ lib/Epub/Epub/css/CssParser.cpp (content)
⚔️ lib/Epub/Epub/css/CssParser.h (content)
⚔️ lib/Epub/Epub/hyphenation/LanguageRegistry.cpp (content)
⚔️ lib/Epub/Epub/hyphenation/LiangHyphenation.cpp (content)
⚔️ lib/Epub/Epub/hyphenation/SerializedHyphenationTrie.h (content)
⚔️ lib/Epub/Epub/hyphenation/generated/hyph-de.trie.h (content)
⚔️ lib/Epub/Epub/hyphenation/generated/hyph-en.trie.h (content)
⚔️ lib/Epub/Epub/hyphenation/generated/hyph-es.trie.h (content)
⚔️ lib/Epub/Epub/hyphenation/generated/hyph-fr.trie.h (content)
⚔️ lib/Epub/Epub/hyphenation/generated/hyph-it.trie.h (content)
⚔️ lib/Epub/Epub/hyphenation/generated/hyph-ru.trie.h (content)
⚔️ lib/Epub/Epub/parsers/ChapterHtmlSlimParser.cpp (content)
⚔️ lib/Epub/Epub/parsers/ChapterHtmlSlimParser.h (content)
⚔️ lib/Epub/Epub/parsers/ContentOpfParser.cpp (content)
⚔️ lib/Epub/Epub/parsers/ContentOpfParser.h (content)
⚔️ lib/GfxRenderer/GfxRenderer.cpp (content)
⚔️ lib/GfxRenderer/GfxRenderer.h (content)
⚔️ lib/ZipFile/ZipFile.cpp (content)
⚔️ platformio.ini (content)
⚔️ scripts/build_html.py (content)
⚔️ scripts/generate_hyphenation_trie.py (content)
⚔️ src/CrossPointSettings.cpp (content)
⚔️ src/CrossPointSettings.h (content)
⚔️ src/RecentBooksStore.cpp (content)
⚔️ src/SettingsList.h (content)
⚔️ src/activities/Activity.h (content)
⚔️ src/activities/ActivityWithSubactivity.cpp (content)
⚔️ src/activities/ActivityWithSubactivity.h (content)
⚔️ src/activities/boot_sleep/BootActivity.cpp (content)
⚔️ src/activities/boot_sleep/SleepActivity.cpp (content)
⚔️ src/activities/browser/OpdsBookBrowserActivity.cpp (content)
⚔️ src/activities/browser/OpdsBookBrowserActivity.h (content)
⚔️ src/activities/home/HomeActivity.cpp (content)
⚔️ src/activities/home/HomeActivity.h (content)
⚔️ src/activities/home/MyLibraryActivity.cpp (content)
⚔️ src/activities/home/MyLibraryActivity.h (content)
⚔️ src/activities/home/RecentBooksActivity.cpp (content)
⚔️ src/activities/home/RecentBooksActivity.h (content)
⚔️ src/activities/network/CalibreConnectActivity.cpp (content)
⚔️ src/activities/network/CalibreConnectActivity.h (content)
⚔️ src/activities/network/CrossPointWebServerActivity.cpp (content)
⚔️ src/activities/network/CrossPointWebServerActivity.h (content)
⚔️ src/activities/network/NetworkModeSelectionActivity.cpp (content)
⚔️ src/activities/network/NetworkModeSelectionActivity.h (content)
⚔️ src/activities/network/WifiSelectionActivity.cpp (content)
⚔️ src/activities/network/WifiSelectionActivity.h (content)
⚔️ src/activities/reader/EpubReaderActivity.cpp (content)
⚔️ src/activities/reader/EpubReaderActivity.h (content)
⚔️ src/activities/reader/EpubReaderChapterSelectionActivity.cpp (content)
⚔️ src/activities/reader/EpubReaderChapterSelectionActivity.h (content)
⚔️ src/activities/reader/EpubReaderMenuActivity.cpp (content)
⚔️ src/activities/reader/EpubReaderMenuActivity.h (content)
⚔️ src/activities/reader/EpubReaderPercentSelectionActivity.cpp (content)
⚔️ src/activities/reader/EpubReaderPercentSelectionActivity.h (content)
⚔️ src/activities/reader/KOReaderSyncActivity.cpp (content)
⚔️ src/activities/reader/KOReaderSyncActivity.h (content)
⚔️ src/activities/reader/ReaderActivity.cpp (content)
⚔️ src/activities/reader/TxtReaderActivity.cpp (content)
⚔️ src/activities/reader/TxtReaderActivity.h (content)
⚔️ src/activities/reader/XtcReaderActivity.cpp (content)
⚔️ src/activities/reader/XtcReaderActivity.h (content)
⚔️ src/activities/reader/XtcReaderChapterSelectionActivity.cpp (content)
⚔️ src/activities/reader/XtcReaderChapterSelectionActivity.h (content)
⚔️ src/activities/settings/ButtonRemapActivity.cpp (content)
⚔️ src/activities/settings/ButtonRemapActivity.h (content)
⚔️ src/activities/settings/CalibreSettingsActivity.cpp (content)
⚔️ src/activities/settings/CalibreSettingsActivity.h (content)
⚔️ src/activities/settings/ClearCacheActivity.cpp (content)
⚔️ src/activities/settings/ClearCacheActivity.h (content)
⚔️ src/activities/settings/KOReaderAuthActivity.cpp (content)
⚔️ src/activities/settings/KOReaderAuthActivity.h (content)
⚔️ src/activities/settings/KOReaderSettingsActivity.cpp (content)
⚔️ src/activities/settings/KOReaderSettingsActivity.h (content)
⚔️ src/activities/settings/OtaUpdateActivity.cpp (content)
⚔️ src/activities/settings/OtaUpdateActivity.h (content)
⚔️ src/activities/settings/SettingsActivity.cpp (content)
⚔️ src/activities/settings/SettingsActivity.h (content)
⚔️ src/activities/util/KeyboardEntryActivity.cpp (content)
⚔️ src/activities/util/KeyboardEntryActivity.h (content)
⚔️ src/components/themes/BaseTheme.cpp (content)
⚔️ src/components/themes/BaseTheme.h (content)
⚔️ src/components/themes/lyra/LyraTheme.cpp (content)
⚔️ src/components/themes/lyra/LyraTheme.h (content)
⚔️ src/main.cpp (content)
⚔️ src/network/CrossPointWebServer.cpp (content)
⚔️ src/network/OtaUpdater.cpp (content)

These conflicts must be resolved before merging into master.
Resolve conflicts locally and push changes to this branch.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title 'perf: Reduce overall flash usage by 30.7% by compressing built-in fonts' accurately and specifically summarizes the main change: font compression to reduce flash usage, with a concrete improvement metric.
Description check ✅ Passed The description thoroughly explains the goals (compress reader font bitmaps), key changes (EpdFontGroup struct, --compress flag, FontDecompressor class, GfxRenderer routing, uzlib usage, font regeneration), and provides detailed context including benchmarks, resource impacts, and AI usage transparency.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

Tip

Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord.


Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
lib/EpdFont/scripts/fontconvert.py (1)

274-418: ⚠️ Potential issue | 🟡 Minor

Fix Ruff lint findings in the compression path.

  • Line 307: packed is unused (B007) → rename to _packed.
  • Lines 377/382: trailing semicolons after print(...) (E703).
  • Lines 417–418: f prefixes without placeholders (F541).
🧹 Suggested lint fixes
-    for i, (props, packed) in enumerate(all_glyphs):
+    for i, (props, _packed) in enumerate(all_glyphs):
@@
-    print ("};\n");
+    print("};\n")
@@
-    print ("};\n");
+    print("};\n")
@@
-    print(f"    nullptr,")
-    print(f"    0,")
+    print("    nullptr,")
+    print("    0,")
🤖 Fix all issues with AI agents
In `@lib/EpdFont/FontDecompressor.cpp`:
- Around line 99-105: The decompression success check is incomplete: after
calling uzlib_uncompress(&decomp) you must validate that the decompressor
actually wrote the expected number of bytes (compare decomp.dest against
dest_limit or expected_size) and optionally that input was fully consumed, and
treat any short output as failure; if the check fails, free outBuf, log the
groupIndex and actual dest value (using Serial.printf similar to the existing
log) and return false. Ensure you reference uzlib_uncompress, the decomp struct,
outBuf, dest_limit/expected_size and groupIndex when making these checks and
cleanup.
- Around line 115-133: The getBitmap function currently assumes compression
groups exist and calls getGroupIndex/decompressGroup even when
fontData->groupCount == 0 or fontData->groups == nullptr; add a fast-path at the
start of FontDecompressor::getBitmap that checks if fontData->groupCount == 0 or
fontData->groups == nullptr and in that case return the direct bitmap pointer
for the glyph (or nullptr if glyph->dataOffset is invalid), bypassing
cache/decompression; reference getBitmap, getGroupIndex, decompressGroup, and
EpdFontData->groupCount/groups when implementing the guard.

In `@lib/EpdFont/scripts/verify_compression.py`:
- Around line 142-150: The loop captures the three-tuple returned by
verify_font_file into (font_name, success, message) but never uses font_name,
causing an unused-variable lint warning; rename font_name to _font_name or _ to
indicate intentional unused, or include the font_name in the print/log output
(e.g., use font_name in the PASS/FAIL messages) so the variable is used; update
the for-loop assignment where verify_font_file is called and adjust any related
prints (still referencing verify_font_file, font_name, success, message).

In `@lib/GfxRenderer/GfxRenderer.cpp`:
- Around line 6-11: In getGlyphBitmap, when fontData->groups is non-null
(compressed font) but fontDecompressor is null you must not index into
fontData->bitmap with glyph->dataOffset; instead guard that case and fail fast
by returning nullptr (or an explicit error) to avoid returning a pointer into
compressed data; update the function GfxRenderer::getGlyphBitmap to check
(fontData->groups != nullptr && fontDecompressor == nullptr) and return nullptr
when true, leaving the existing path for decompressed fonts and the
fontDecompressor path intact so callers (rendering code) can handle the null
result safely.

In `@lib/uzlib/src/tinflate.c`:
- Around line 662-669: The ISIZE trailer is not validated: when handling
TINF_CHKSUM_CRC after reading the CRC with tinf_get_le_uint32(d) you must also
verify the 4-byte ISIZE against a running total of decompressed bytes. Add a
uint32/uint64 field to the decompressor state (e.g., d->total_out or
d->out_count), increment it wherever output bytes are written in the inflate
paths, then after val = tinf_get_le_uint32(d) compare (uint32) d->total_out to
val and return TINF_CHKSUM_ERROR on mismatch; update any initialization/reset
logic for the state and ensure the comparison uses 32-bit wrapping semantics to
match gzip ISIZE.
- Around line 482-491: The memcpy call that copies LZ77 back-reference data
(using variables d->curlen, d->dest, d->dest_limit and d->lzOff) can operate on
overlapping source/destination ranges; replace memcpy(d->dest, d->dest +
d->lzOff, to_copy) with memmove to guarantee correct behavior for overlapping
copies, and ensure <string.h> is included if memmove isn't already available in
this compilation unit.
- Around line 193-205: The pointer comparison in uzlib_get_byte is unsafe when
d->source_limit can be NULL; update the initial guard to check d->source_limit
before doing the relational comparison (i.e., only evaluate d->source <
d->source_limit when d->source_limit is non-NULL), so the function first tests
if (d->source_limit && d->source < d->source_limit) and otherwise falls back to
the d->readSource / d->eof branch (using the existing d->readSource callback and
d->eof logic); ensure the behavior matches the struct comment about streaming
mode when source_limit == NULL.
🧹 Nitpick comments (8)
docs/font-compression.md (1)

3-35: Add measurement context for the reported metrics.

The tables are useful, but they’re hard to reproduce without stating the target (device/emulator), build profile/toolchain, measurement command, date, and what the percentages are relative to. A short context section would prevent misinterpretation.

✍️ Suggested doc addition
 ## Flash & RAM
+
+### Measurement context
+- Target: <device/emulator>
+- Build profile/toolchain: <profile + version>
+- Command/script: <how sizes were measured>
+- Date/commit: <timestamp or commit SHA>
+- Percentages: relative to <configured app partition size>
 
 | | baseline | font-compression | Difference |
lib/EpdFont/EpdFontData.h (1)

18-45: Document or widen uint16_t limits on glyph and group counts.

glyphCount, firstGlyphIndex, and groupCount use uint16_t, which caps fonts at 65,535 glyphs per group/total. Current fonts reach only 877 glyphs, well below the limit. However, no assertions or documentation enforce this constraint. For future fonts, either document this limit explicitly or widen these fields to uint32_t to avoid overflow risks.

src/activities/reader/EpubReaderActivity.cpp (1)

648-650: Add a short rationale for per‑page font‑cache clears

If the cache is intentionally page‑scoped (heap/fragmentation control), a brief comment will prevent accidental removal later.

♻️ Suggested comment
-    const auto start = millis();
-    renderer.clearFontCache();
+    const auto start = millis();
+    // Clear per page to limit heap usage and avoid stale decompressed glyphs.
+    renderer.clearFontCache();
lib/uzlib/src/tinf.h (1)

1-3: Add a header guard / pragma once

Even for a thin compatibility header, a guard prevents accidental multiple inclusion if it expands later.

♻️ Suggested update
+#pragma once
 /* Compatibility header for the original tinf lib/older versions of uzlib.
    Note: may be removed in the future, please migrate to uzlib.h. */
 `#include` "uzlib.h"
lib/uzlib/src/tinf_compat.h (1)

1-9: Add an include guard/pragma once for this new compatibility header.

Without a guard, repeated includes can re-define these macros and trigger warnings on some toolchains. Consider adding #pragma once (or a classic guard) at the top.

♻️ Proposed tweak
+#pragma once
+
 /* This header contains compatibility defines for the original tinf API
    and uzlib 2.x and below API. These defines are deprecated and going
    to be removed in the future, so applications should migrate to new
    uzlib API. */
lib/ZipFile/ZipFile.cpp (1)

9-29: Validate inflated size to catch corrupt ZIPs.

Line 21: tinfl_decompress can return DONE even if the produced byte count doesn’t match the expected inflatedSize. Checking outBytes (and optionally inBytes) avoids returning partially filled buffers.

✅ Proposed check
   const tinfl_status status = tinfl_decompress(inflator, inputBuf, &inBytes, nullptr, outputBuf, &outBytes,
                                                TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF);

   free(inflator);

-  if (status != TINFL_STATUS_DONE) {
+  if (status != TINFL_STATUS_DONE || outBytes != inflatedSize || inBytes != deflatedSize) {
     LOG_ERR("ZIP", "tinfl_decompress() failed with status %d", status);
     return false;
   }
lib/uzlib/src/defl_static.h (1)

1-37: Add an include guard/pragma once for this new header.

This prevents duplicate prototype declarations if included multiple times in a TU.

♻️ Proposed tweak
+#pragma once
+
 /*
  * Copyright (c) uzlib authors
  *
lib/EpdFont/FontDecompressor.h (1)

9-35: Make FontDecompressor non-copyable to avoid shallow pointer copies.

This class owns raw buffers in CacheEntry. Deleting copy/assign prevents accidental double-free or corruption.

♻️ Proposed safeguard
 class FontDecompressor {
  public:
+  FontDecompressor() = default;
+  FontDecompressor(const FontDecompressor&) = delete;
+  FontDecompressor& operator=(const FontDecompressor&) = delete;
   bool init();
   void deinit();

Copy link
Member

@daveallie daveallie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few things from CodeRabbit to address and just some cleanup of Serial.printf needed.

- GfxRenderer.cpp:6-16 -- Guard compressed fonts when no decompressor is set. Now logs an error and returns nullptr instead of indexing into compressed bitmap data.
- FontDecompressor.cpp:100 -- Validate decompression produced expected byte count by checking decomp.dest != decomp.dest_limit.
- FontDecompressor.cpp:114-117 -- Guard for fonts without compression groups -- fast path returning direct bitmap pointer.
- verify_compression.py:144 -- Prefix unused font_name with _ to fix RUF059 lint warning.
- FontDecompressor.cpp:3,84,101 -- Replaced #include <HardwareSerial.h> with #include <Logging.h> and swapped both Serial.printf(...) calls with LOG_ERR(...) macros.
- Deleted docs/font-compression-benchmark.md -- Same findings have been added to the PR description.
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Fix all issues with AI agents
Verify each finding against the current code and only fix it if needed.


In `@lib/EpdFont/FontDecompressor.cpp`:
- Around line 35-43: The getGroupIndex function currently returns 0 when no
matching group is found which can cause wrong group selection and OOB reads;
change getGroupIndex to return a sentinel (e.g., UINT16_MAX) when no group
matches, and update callers (notably getBitmap and the logic around the group
lookup used in the 114-135 range) to check for that sentinel and bail out early
(return an error/empty bitmap or safe default) rather than proceeding with group
0; reference getGroupIndex and getBitmap so you locate and add the defensive
check where the group index is used.
- Around line 14-33: Extract the duplicated cleanup logic in
FontDecompressor::deinit and FontDecompressor::clearCache into a private helper
(e.g., clearCacheEntries or freeCacheEntries) that iterates over cache, frees
entry.data, sets entry.data = nullptr and entry.valid = false; then have deinit
call that helper and clearCache call the helper and also reset accessCounter =
0. Update references to cache and accessCounter in those methods accordingly so
behavior stays identical and duplication is removed.

In `@lib/EpdFont/scripts/verify_compression.py`:
- Around line 81-92: The script currently proceeds when a Groups array is found
but parse_groups returns an empty list, causing silent false positives; add a
guard after calling parse_groups(groups_array_match.group(1)) to check if the
returned groups is empty and treat that as an error (return (font_name, False,
"no groups parsed") or similar). Specifically update the block using
groups_array_match, parse_groups, and the groups variable to fail fast when
groups is empty so downstream logic that expects group entries cannot continue.

In `@lib/GfxRenderer/GfxRenderer.cpp`:
- Around line 6-16: The getGlyphBitmap path detects compressed fonts
(fontData->groups != nullptr) and currently calls LOG_ERR("GFX", "Compressed
font but no FontDecompressor set") every glyph when fontDecompressor is null;
add a log-once guard (e.g. a static or GfxRenderer member boolean like
decompressorMissingLogged) checked and set inside GfxRenderer::getGlyphBitmap
before calling LOG_ERR so the error is emitted only once per process or renderer
instance, leaving the existing return nullptr behavior unchanged.
🧹 Nitpick comments (2)
🤖 Fix all nitpicks with AI agents
Verify each finding against the current code and only fix it if needed.


In `@lib/EpdFont/FontDecompressor.cpp`:
- Around line 14-33: Extract the duplicated cleanup logic in
FontDecompressor::deinit and FontDecompressor::clearCache into a private helper
(e.g., clearCacheEntries or freeCacheEntries) that iterates over cache, frees
entry.data, sets entry.data = nullptr and entry.valid = false; then have deinit
call that helper and clearCache call the helper and also reset accessCounter =
0. Update references to cache and accessCounter in those methods accordingly so
behavior stays identical and duplication is removed.

In `@lib/GfxRenderer/GfxRenderer.cpp`:
- Around line 6-16: The getGlyphBitmap path detects compressed fonts
(fontData->groups != nullptr) and currently calls LOG_ERR("GFX", "Compressed
font but no FontDecompressor set") every glyph when fontDecompressor is null;
add a log-once guard (e.g. a static or GfxRenderer member boolean like
decompressorMissingLogged) checked and set inside GfxRenderer::getGlyphBitmap
before calling LOG_ERR so the error is emitted only once per process or renderer
instance, leaving the existing return nullptr behavior unchanged.
lib/EpdFont/FontDecompressor.cpp (1)

14-33: Consider deduplicating cache cleanup.

deinit() and clearCache() share identical cleanup logic. A small helper would reduce duplication and keep behavior consistent if you tweak it later.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/EpdFont/FontDecompressor.cpp` around lines 14 - 33, Extract the
duplicated cleanup logic in FontDecompressor::deinit and
FontDecompressor::clearCache into a private helper (e.g., clearCacheEntries or
freeCacheEntries) that iterates over cache, frees entry.data, sets entry.data =
nullptr and entry.valid = false; then have deinit call that helper and
clearCache call the helper and also reset accessCounter = 0. Update references
to cache and accessCounter in those methods accordingly so behavior stays
identical and duplication is removed.
lib/GfxRenderer/GfxRenderer.cpp (1)

6-16: Consider rate‑limiting the “no decompressor” error.

If a compressed font is rendered without a decompressor, this logs once per glyph and can flood logs / slow rendering. A log‑once guard keeps the error visible without spamming.

🔇 Log-once guard
   if (fontData->groups != nullptr) {
     if (!fontDecompressor) {
-      LOG_ERR("GFX", "Compressed font but no FontDecompressor set");
+      static bool logged = false;
+      if (!logged) {
+        LOG_ERR("GFX", "Compressed font but no FontDecompressor set");
+        logged = true;
+      }
       return nullptr;
     }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/GfxRenderer/GfxRenderer.cpp` around lines 6 - 16, The getGlyphBitmap path
detects compressed fonts (fontData->groups != nullptr) and currently calls
LOG_ERR("GFX", "Compressed font but no FontDecompressor set") every glyph when
fontDecompressor is null; add a log-once guard (e.g. a static or GfxRenderer
member boolean like decompressorMissingLogged) checked and set inside
GfxRenderer::getGlyphBitmap before calling LOG_ERR so the error is emitted only
once per process or renderer instance, leaving the existing return nullptr
behavior unchanged.
📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 489fdf9 and f371343.

📒 Files selected for processing (3)
  • lib/EpdFont/FontDecompressor.cpp
  • lib/EpdFont/scripts/verify_compression.py
  • lib/GfxRenderer/GfxRenderer.cpp
🧰 Additional context used
🧬 Code graph analysis (2)
lib/GfxRenderer/GfxRenderer.cpp (3)
src/activities/boot_sleep/SleepActivity.cpp (4)
  • bitmap (57-57)
  • bitmap (81-81)
  • bitmap (96-96)
  • bitmap (265-265)
src/components/themes/BaseTheme.cpp (1)
  • bitmap (313-313)
src/components/themes/lyra/LyraTheme.cpp (1)
  • bitmap (287-287)
lib/EpdFont/FontDecompressor.cpp (1)
lib/uzlib/src/tinflate.c (2)
  • uzlib_uncompress_init (552-562)
  • uzlib_uncompress (565-626)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: build
🔇 Additional comments (10)
lib/EpdFont/scripts/verify_compression.py (4)

15-18: Looks good: concise hex extraction.


21-32: Clear group parsing logic.


35-48: Glyph parsing is straightforward.


127-158: CLI flow is clean and deterministic.

lib/EpdFont/FontDecompressor.cpp (4)

9-12: Init is clean and minimal.


45-52: Cache lookup is clear.


54-69: LRU eviction logic looks good.


71-112: Decompression path is solid.

lib/GfxRenderer/GfxRenderer.cpp (2)

806-824: Good consolidation of font data access.


971-978: LGTM for centralized bitmap retrieval.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Fix all issues with AI agents
Verify each finding against the current code and only fix it if needed.


In `@lib/EpdFont/FontDecompressor.cpp`:
- Around line 14-33: Both FontDecompressor::deinit and
FontDecompressor::clearCache duplicate the same loop that frees cache entries;
extract that logic into a private helper (e.g.,
FontDecompressor::freeAllEntries) which iterates over cache, frees entry.data
and sets entry.data = nullptr and entry.valid = false, then call
freeAllEntries() from both deinit() and clearCache(); ensure clearCache still
resets accessCounter = 0 after calling the helper.

In `@lib/EpdFont/scripts/verify_compression.py`:
- Around line 15-18: The parse_hex_array function's regex only matches '0x' with
exactly two hex digits so it will miss variants like '0X' or single-digit bytes
such as '0xA'; update the pattern used in parse_hex_array to accept either '0x'
or '0X' and 1–2 hex digits (e.g. use a pattern like r'0[xX]([0-9A-Fa-f]{1,2})'),
keep converting each capture with int(h, 16) and returning bytes(...) so
single-digit captures are parsed correctly into their numeric byte values.
🧹 Nitpick comments (2)
🤖 Fix all nitpicks with AI agents
Verify each finding against the current code and only fix it if needed.


In `@lib/EpdFont/FontDecompressor.cpp`:
- Around line 14-33: Both FontDecompressor::deinit and
FontDecompressor::clearCache duplicate the same loop that frees cache entries;
extract that logic into a private helper (e.g.,
FontDecompressor::freeAllEntries) which iterates over cache, frees entry.data
and sets entry.data = nullptr and entry.valid = false, then call
freeAllEntries() from both deinit() and clearCache(); ensure clearCache still
resets accessCounter = 0 after calling the helper.

In `@lib/EpdFont/scripts/verify_compression.py`:
- Around line 15-18: The parse_hex_array function's regex only matches '0x' with
exactly two hex digits so it will miss variants like '0X' or single-digit bytes
such as '0xA'; update the pattern used in parse_hex_array to accept either '0x'
or '0X' and 1–2 hex digits (e.g. use a pattern like r'0[xX]([0-9A-Fa-f]{1,2})'),
keep converting each capture with int(h, 16) and returning bytes(...) so
single-digit captures are parsed correctly into their numeric byte values.
lib/EpdFont/FontDecompressor.cpp (1)

14-33: Consider extracting shared cache-clearing logic.

deinit() and clearCache() share identical loops for freeing cache entries. You could extract a private helper (e.g., freeAllEntries()) to reduce duplication.

♻️ Proposed refactor
+void FontDecompressor::freeAllEntries() {
+  for (auto& entry : cache) {
+    if (entry.data) {
+      free(entry.data);
+      entry.data = nullptr;
+    }
+    entry.valid = false;
+  }
+}
+
 void FontDecompressor::deinit() {
-  for (auto& entry : cache) {
-    if (entry.data) {
-      free(entry.data);
-      entry.data = nullptr;
-    }
-    entry.valid = false;
-  }
+  freeAllEntries();
 }
 
 void FontDecompressor::clearCache() {
-  for (auto& entry : cache) {
-    if (entry.data) {
-      free(entry.data);
-      entry.data = nullptr;
-    }
-    entry.valid = false;
-  }
+  freeAllEntries();
   accessCounter = 0;
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/EpdFont/FontDecompressor.cpp` around lines 14 - 33, Both
FontDecompressor::deinit and FontDecompressor::clearCache duplicate the same
loop that frees cache entries; extract that logic into a private helper (e.g.,
FontDecompressor::freeAllEntries) which iterates over cache, frees entry.data
and sets entry.data = nullptr and entry.valid = false, then call
freeAllEntries() from both deinit() and clearCache(); ensure clearCache still
resets accessCounter = 0 after calling the helper.
lib/EpdFont/scripts/verify_compression.py (1)

15-18: Make hex parsing tolerant of 0X or single‑digit bytes.

Optional robustness: if the generator ever emits 0X or 0xA, the current regex would drop bytes. Broadening the pattern avoids false negatives with minimal risk.

♻️ Suggested tweak
-    hex_vals = re.findall(r'0x([0-9A-Fa-f]{2})', text)
+    hex_vals = re.findall(r'0[xX]([0-9A-Fa-f]{1,2})', text)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/EpdFont/scripts/verify_compression.py` around lines 15 - 18, The
parse_hex_array function's regex only matches '0x' with exactly two hex digits
so it will miss variants like '0X' or single-digit bytes such as '0xA'; update
the pattern used in parse_hex_array to accept either '0x' or '0X' and 1–2 hex
digits (e.g. use a pattern like r'0[xX]([0-9A-Fa-f]{1,2})'), keep converting
each capture with int(h, 16) and returning bytes(...) so single-digit captures
are parsed correctly into their numeric byte values.
📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f371343 and 3f6a97c.

📒 Files selected for processing (2)
  • lib/EpdFont/FontDecompressor.cpp
  • lib/EpdFont/scripts/verify_compression.py
🧰 Additional context used
🧬 Code graph analysis (1)
lib/EpdFont/FontDecompressor.cpp (2)
lib/Epub/Epub/BookMetadataCache.cpp (1)
  • entry (358-358)
lib/uzlib/src/tinflate.c (2)
  • uzlib_uncompress_init (552-562)
  • uzlib_uncompress (565-626)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: build
🔇 Additional comments (7)
lib/EpdFont/FontDecompressor.cpp (4)

9-12: LGTM!

Clean initialization of the decompression state.


35-69: LGTM!

The sentinel return for missing groups and the LRU eviction logic are both well-implemented. Linear search is appropriate for the small cache size.


71-112: LGTM!

The decompression logic is sound. The validation at line 100 correctly checks both the return status and that the full output buffer was filled. Memory management handles both success and failure paths properly.


114-140: The offset semantics in the C++ code are handled correctly by fontconvert.py.

When compression is enabled, fontconvert.py (lines 326-357) explicitly converts dataOffset from absolute offsets to within-group offsets. The comment on line 334 states: "Update glyph's dataOffset to be within-group offset". Each glyph's dataOffset is recalculated as len(group_data) (line 335), representing its position within the group's concatenated bitmap data, not the global bitmap.

This matches the C++ expectations:

  • Uncompressed (line 116): dataOffset is absolute into fontData->bitmap
  • Compressed (lines 129, 139): dataOffset is relative to entry->data (the decompressed group)

The implementation is correct.

lib/EpdFont/scripts/verify_compression.py (3)

21-48: Parsing helpers look solid.

Clear structure and direct mapping to the header fields; nice and readable.


51-126: Verification flow is thorough and well‑guarded.

Good early exits and detailed error messages make failures actionable.


129-164: CLI flow and exit behavior look good.

Straightforward usage checks, summary, and proper non‑zero exit on failures.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Fix all issues with AI agents
Verify each finding against the current code and only fix it if needed.


In `@lib/EpdFont/FontDecompressor.cpp`:
- Around line 110-136: In FontDecompressor::getBitmap add defensive bounds
checks for glyph->dataOffset before indexing into fontData->bitmap and
entry->data: ensure glyph->dataOffset is less than the corresponding buffer size
(for non-grouped case, compare against fontData->bitmap length; for
cached/decompressed case, compare against the decompressed entry->size or buffer
length in CacheEntry) and if out-of-range log an error (e.g., via LOG_ERR) and
return nullptr; adjust checks around the findInCache and decompressGroup paths
(functions findInCache, decompressGroup, and the CacheEntry structure) so you
never access entry->data[glyph->dataOffset] or
&fontData->bitmap[glyph->dataOffset] without validating the offset.
🧹 Nitpick comments (1)
🤖 Fix all nitpicks with AI agents
Verify each finding against the current code and only fix it if needed.


In `@lib/EpdFont/FontDecompressor.cpp`:
- Around line 110-136: In FontDecompressor::getBitmap add defensive bounds
checks for glyph->dataOffset before indexing into fontData->bitmap and
entry->data: ensure glyph->dataOffset is less than the corresponding buffer size
(for non-grouped case, compare against fontData->bitmap length; for
cached/decompressed case, compare against the decompressed entry->size or buffer
length in CacheEntry) and if out-of-range log an error (e.g., via LOG_ERR) and
return nullptr; adjust checks around the findInCache and decompressGroup paths
(functions findInCache, decompressGroup, and the CacheEntry structure) so you
never access entry->data[glyph->dataOffset] or
&fontData->bitmap[glyph->dataOffset] without validating the offset.
lib/EpdFont/FontDecompressor.cpp (1)

110-136: Consider bounds-checking glyph->dataOffset for defense-in-depth.

Lines 125 and 135 access entry->data[glyph->dataOffset] without verifying that dataOffset is within the decompressed buffer size. While the font generation tooling and verify_compression.py should ensure consistency, a bounds check would guard against corrupt or mismatched metadata.

Given this is a performance-sensitive path and the data is tooling-generated, this is optional.

🛡️ Optional defensive bounds check
   // Check cache
   CacheEntry* entry = findInCache(fontData, groupIndex);
   if (entry) {
     entry->lastUsed = ++accessCounter;
+    if (glyph->dataOffset >= entry->dataSize) {
+      LOG_ERR("FDC", "dataOffset %u out of bounds for group %u (size %u)", glyph->dataOffset, groupIndex, entry->dataSize);
+      return nullptr;
+    }
     return &entry->data[glyph->dataOffset];
   }

   // Cache miss - decompress
   entry = findEvictionCandidate();
   if (!decompressGroup(fontData, groupIndex, entry)) {
     return nullptr;
   }

   entry->lastUsed = ++accessCounter;
+  if (glyph->dataOffset >= entry->dataSize) {
+    LOG_ERR("FDC", "dataOffset %u out of bounds for group %u (size %u)", glyph->dataOffset, groupIndex, entry->dataSize);
+    return nullptr;
+  }
   return &entry->data[glyph->dataOffset];
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/EpdFont/FontDecompressor.cpp` around lines 110 - 136, In
FontDecompressor::getBitmap add defensive bounds checks for glyph->dataOffset
before indexing into fontData->bitmap and entry->data: ensure glyph->dataOffset
is less than the corresponding buffer size (for non-grouped case, compare
against fontData->bitmap length; for cached/decompressed case, compare against
the decompressed entry->size or buffer length in CacheEntry) and if out-of-range
log an error (e.g., via LOG_ERR) and return nullptr; adjust checks around the
findInCache and decompressGroup paths (functions findInCache, decompressGroup,
and the CacheEntry structure) so you never access entry->data[glyph->dataOffset]
or &fontData->bitmap[glyph->dataOffset] without validating the offset.
📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3f6a97c and 7c66978.

📒 Files selected for processing (2)
  • lib/EpdFont/FontDecompressor.cpp
  • lib/EpdFont/FontDecompressor.h
🧰 Additional context used
🧬 Code graph analysis (2)
lib/EpdFont/FontDecompressor.cpp (1)
lib/uzlib/src/tinflate.c (2)
  • uzlib_uncompress_init (552-562)
  • uzlib_uncompress (565-626)
lib/EpdFont/FontDecompressor.h (1)
lib/EpdFont/FontDecompressor.cpp (18)
  • init (9-12)
  • init (9-9)
  • deinit (24-24)
  • deinit (24-24)
  • getBitmap (110-136)
  • getBitmap (110-110)
  • clearCache (26-29)
  • clearCache (26-26)
  • freeAllEntries (14-22)
  • freeAllEntries (14-14)
  • getGroupIndex (31-39)
  • getGroupIndex (31-31)
  • findInCache (41-48)
  • findInCache (41-41)
  • findEvictionCandidate (50-65)
  • findEvictionCandidate (50-50)
  • decompressGroup (67-108)
  • decompressGroup (67-67)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: build
🔇 Additional comments (6)
lib/EpdFont/FontDecompressor.h (1)

1-42: LGTM!

The header is well-structured with a clear public API and appropriate documentation about pointer lifetime. The 4-slot LRU cache design aligns with the PR's stated goals for balancing memory usage and cache hit rates.

lib/EpdFont/FontDecompressor.cpp (5)

1-12: LGTM!

Clean initialization approach - memset ensures a known state for the uzlib_uncomp structure before use.


14-29: LGTM!

Proper memory management with null-after-free pattern to prevent double-free issues.


31-39: LGTM!

Linear search is appropriate for the small number of groups. The sentinel return value for not-found cases properly guards against invalid access.


41-65: LGTM!

The cache lookup and LRU eviction logic is straightforward and correct for a 4-slot cache. Preferring invalid slots before evicting valid entries is the right approach.


67-108: LGTM!

The decompression flow is well-structured with proper error handling and cleanup. The validation at line 96 correctly verifies both the return status and that the expected byte count was produced.

@adriancaruana
Copy link
Contributor Author

@CodeRabbit help

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 17, 2026

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

CodeRabbit commands

These commands are invoked using PR/Issue comments.

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai evaluate custom pre-merge check --instructions <custom-checks-instructions> --name <custom-checks-title> [--mode <error|warning>] to validate the custom pre-merge checks instructions. Defaults to error when --mode is omitted.
    • @coderabbitai ignore pre-merge checks to override pre-merge checks and get an approval on PR.
    • @coderabbitai run pre-merge checks to run pre-merge checks on the pull request.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve to resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai generate configuration to create a PR that adds the current resolved configuration as .coderabbit.yaml (or show it if already present).
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore or @coderabbit ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit configuration file (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • You can also validate your configuration using the online YAML validator.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, support, documentation and community

  • Visit our status page to check the current availability of CodeRabbit.
  • Create a ticket on our support page for assistance with any issues or questions.
  • Visit our documentation site for detailed information on how to use CodeRabbit.
  • Join our Discord community to connect with other users and get help from the community.
  • Follow us on X/Twitter for updates and announcements.

@adriancaruana
Copy link
Contributor Author

@coderabbitai review

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 17, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces on-device DEFLATE decompression for built-in reader fonts and updates the font generation pipeline to emit grouped, compressed bitmap data, targeting a large firmware flash reduction.

Changes:

  • Extend font data format to support compressed bitmap “groups” and regenerate reader fonts using a new --compress option in fontconvert.py.
  • Add FontDecompressor (4-slot LRU) backed by uzlib, and update GfxRenderer to fetch glyph bitmaps via decompression when needed.
  • Clear the decompression cache between page renders in reader activities to bound RAM usage.

Reviewed changes

Copilot reviewed 24 out of 73 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
src/main.cpp Instantiates/initializes FontDecompressor and wires it into GfxRenderer.
src/activities/reader/TxtReaderActivity.cpp Clears font decompression cache each page render.
src/activities/reader/EpubReaderActivity.cpp Clears font decompression cache each page render.
lib/GfxRenderer/GfxRenderer.h Adds decompressor hook and clearFontCache() API.
lib/GfxRenderer/GfxRenderer.cpp Routes glyph bitmap access through getGlyphBitmap() (compressed/uncompressed).
lib/EpdFont/EpdFontData.h Adds EpdFontGroup + groups/groupCount fields to EpdFontData.
lib/EpdFont/FontDecompressor.h Declares FontDecompressor and its 4-slot LRU cache.
lib/EpdFont/FontDecompressor.cpp Implements uzlib-backed group decompression + caching.
lib/EpdFont/scripts/fontconvert.py Adds --compress and grouping + raw-DEFLATE compression output.
lib/EpdFont/scripts/verify_compression.py Adds a script to validate compressed font headers.
lib/EpdFont/scripts/convert-builtin-fonts.sh Enables compression for reader fonts and runs verification.
lib/EpdFont/builtinFonts/*.h Updates generated font headers for new EpdFontData fields (and regenerates reader fonts with compression).
lib/uzlib/** Adds the uzlib decompression library sources/config.
lib/uzlib/library.json Declares uzlib as a PlatformIO library.
lib/ZipFile/ZipFile.cpp Minor refactor/formatting around one-shot inflate helper usage.
bin/clang-format-fix Excludes vendored lib/uzlib/ from clang-format script.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- Add defensive bounds check for dataLength in addition to dataOffset
- Add cache clearing to init() to ensure a clean state when reinitializing
- Clear cache at end of page-render, not before
@adriancaruana
Copy link
Contributor Author

@daveallie @osteotek I think this is ready for a final look.

@daveallie daveallie merged commit 47aa0dd into crosspoint-reader:master Feb 19, 2026
6 checks passed
ariel-lindemann pushed a commit to ariel-lindemann/crosspoint-reader that referenced this pull request Feb 19, 2026
…ts (crosspoint-reader#831)

## Summary

**What is the goal of this PR?**

Compress reader font bitmaps to reduce flash usage by 30.7%.

**What changes are included?**

- New `EpdFontGroup` struct and extended `EpdFontData` with
`groups`/`groupCount` fields
- `--compress` flag in `fontconvert.py`: groups glyphs (ASCII base group
+ groups of 8) and compresses each with raw DEFLATE
- `FontDecompressor` class with 4-slot LRU cache for on-demand
decompression during rendering
- `GfxRenderer` transparently routes bitmap access through
`getGlyphBitmap()` (compressed or direct flash)
- Uses `uzlib` for decompression with minimal heap overhead.
- 48 reader fonts (Bookerly, NotoSans 12-18pt, OpenDyslexic) regenerated
with compression; 5 UI fonts unchanged
- Round-trip verification script (`verify_compression.py`) runs as part
of font generation
## Additional Context

## Flash & RAM

| | baseline | font-compression | Difference |
|--|--------|-----------------|------------|
| Flash (ELF) | 6,302,476 B (96.2%) | 4,365,022 B (66.6%) | -1,937,454 B
(-30.7%) |
| firmware.bin | 6,468,192 B | 4,531,008 B | -1,937,184 B (-29.9%) |
| RAM | 101,700 B (31.0%) | 103,076 B (31.5%) | +1,376 B (+0.5%) |

## Script-Based Grouping (Cold Cache)

Comparison of uncompressed baseline vs script-based group compression
(4-slot LRU cache, cleared each page). Glyphs are grouped by Unicode
block (ASCII, Latin-1, Latin Extended-A, Combining Marks, Cyrillic,
General Punctuation, etc.) instead of sequential groups of 8.

### Render Time

| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Median** | 414.9 ms | 431.6 ms | +16.7 ms (+4.0%) |
| **Pages** | 37 | 37 | |

### Memory Usage

| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Heap free (median)** | 187.0 KB | 176.3 KB | -10.7 KB |
| **Heap free (min)** | 186.0 KB | 166.5 KB | -19.5 KB |
| **Largest block (median)** | 148.0 KB | 128.0 KB | -20.0 KB |
| **Largest block (min)** | 148.0 KB | 120.0 KB | -28.0 KB |

### Cache Effectiveness

| | Misses/page | Hit rate |
|---|---|---|
| **Compressed (cold cache)** | 2.1 | 99.85% |

------

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

  Did you use AI tools to help write this code? _**YES**_
Implementation was done by Claude Code (Opus 4.6) based on a plan
developed collaboratively. All generated font headers were verified with
an automated round-trip decompression test. The firmware was compiled
successfully but has not yet been tested on-device.

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
saslv pushed a commit to saslv/crosspoint-reader that referenced this pull request Feb 19, 2026
…ts (crosspoint-reader#831)

## Summary

**What is the goal of this PR?**

Compress reader font bitmaps to reduce flash usage by 30.7%.

**What changes are included?**

- New `EpdFontGroup` struct and extended `EpdFontData` with
`groups`/`groupCount` fields
- `--compress` flag in `fontconvert.py`: groups glyphs (ASCII base group
+ groups of 8) and compresses each with raw DEFLATE
- `FontDecompressor` class with 4-slot LRU cache for on-demand
decompression during rendering
- `GfxRenderer` transparently routes bitmap access through
`getGlyphBitmap()` (compressed or direct flash)
- Uses `uzlib` for decompression with minimal heap overhead.
- 48 reader fonts (Bookerly, NotoSans 12-18pt, OpenDyslexic) regenerated
with compression; 5 UI fonts unchanged
- Round-trip verification script (`verify_compression.py`) runs as part
of font generation
## Additional Context

## Flash & RAM

| | baseline | font-compression | Difference |
|--|--------|-----------------|------------|
| Flash (ELF) | 6,302,476 B (96.2%) | 4,365,022 B (66.6%) | -1,937,454 B
(-30.7%) |
| firmware.bin | 6,468,192 B | 4,531,008 B | -1,937,184 B (-29.9%) |
| RAM | 101,700 B (31.0%) | 103,076 B (31.5%) | +1,376 B (+0.5%) |

## Script-Based Grouping (Cold Cache)

Comparison of uncompressed baseline vs script-based group compression
(4-slot LRU cache, cleared each page). Glyphs are grouped by Unicode
block (ASCII, Latin-1, Latin Extended-A, Combining Marks, Cyrillic,
General Punctuation, etc.) instead of sequential groups of 8.

### Render Time

| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Median** | 414.9 ms | 431.6 ms | +16.7 ms (+4.0%) |
| **Pages** | 37 | 37 | |

### Memory Usage

| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Heap free (median)** | 187.0 KB | 176.3 KB | -10.7 KB |
| **Heap free (min)** | 186.0 KB | 166.5 KB | -19.5 KB |
| **Largest block (median)** | 148.0 KB | 128.0 KB | -20.0 KB |
| **Largest block (min)** | 148.0 KB | 120.0 KB | -28.0 KB |

### Cache Effectiveness

| | Misses/page | Hit rate |
|---|---|---|
| **Compressed (cold cache)** | 2.1 | 99.85% |

------

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

  Did you use AI tools to help write this code? _**YES**_
Implementation was done by Claude Code (Opus 4.6) based on a plan
developed collaboratively. All generated font headers were verified with
an automated round-trip decompression test. The firmware was compiled
successfully but has not yet been tested on-device.

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
el pushed a commit to el/crosspoint-reader that referenced this pull request Feb 19, 2026
…ts (crosspoint-reader#831)

## Summary

**What is the goal of this PR?**

Compress reader font bitmaps to reduce flash usage by 30.7%.

**What changes are included?**

- New `EpdFontGroup` struct and extended `EpdFontData` with
`groups`/`groupCount` fields
- `--compress` flag in `fontconvert.py`: groups glyphs (ASCII base group
+ groups of 8) and compresses each with raw DEFLATE
- `FontDecompressor` class with 4-slot LRU cache for on-demand
decompression during rendering
- `GfxRenderer` transparently routes bitmap access through
`getGlyphBitmap()` (compressed or direct flash)
- Uses `uzlib` for decompression with minimal heap overhead.
- 48 reader fonts (Bookerly, NotoSans 12-18pt, OpenDyslexic) regenerated
with compression; 5 UI fonts unchanged
- Round-trip verification script (`verify_compression.py`) runs as part
of font generation
## Additional Context

## Flash & RAM

| | baseline | font-compression | Difference |
|--|--------|-----------------|------------|
| Flash (ELF) | 6,302,476 B (96.2%) | 4,365,022 B (66.6%) | -1,937,454 B
(-30.7%) |
| firmware.bin | 6,468,192 B | 4,531,008 B | -1,937,184 B (-29.9%) |
| RAM | 101,700 B (31.0%) | 103,076 B (31.5%) | +1,376 B (+0.5%) |

## Script-Based Grouping (Cold Cache)

Comparison of uncompressed baseline vs script-based group compression
(4-slot LRU cache, cleared each page). Glyphs are grouped by Unicode
block (ASCII, Latin-1, Latin Extended-A, Combining Marks, Cyrillic,
General Punctuation, etc.) instead of sequential groups of 8.

### Render Time

| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Median** | 414.9 ms | 431.6 ms | +16.7 ms (+4.0%) |
| **Pages** | 37 | 37 | |

### Memory Usage

| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Heap free (median)** | 187.0 KB | 176.3 KB | -10.7 KB |
| **Heap free (min)** | 186.0 KB | 166.5 KB | -19.5 KB |
| **Largest block (median)** | 148.0 KB | 128.0 KB | -20.0 KB |
| **Largest block (min)** | 148.0 KB | 120.0 KB | -28.0 KB |

### Cache Effectiveness

| | Misses/page | Hit rate |
|---|---|---|
| **Compressed (cold cache)** | 2.1 | 99.85% |

------

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

  Did you use AI tools to help write this code? _**YES**_
Implementation was done by Claude Code (Opus 4.6) based on a plan
developed collaboratively. All generated font headers were verified with
an automated round-trip decompression test. The firmware was compiled
successfully but has not yet been tested on-device.

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
lukestein pushed a commit to lukestein/crosspoint-reader that referenced this pull request Feb 20, 2026
…ts (crosspoint-reader#831)

## Summary

**What is the goal of this PR?**

Compress reader font bitmaps to reduce flash usage by 30.7%.

**What changes are included?**

- New `EpdFontGroup` struct and extended `EpdFontData` with
`groups`/`groupCount` fields
- `--compress` flag in `fontconvert.py`: groups glyphs (ASCII base group
+ groups of 8) and compresses each with raw DEFLATE
- `FontDecompressor` class with 4-slot LRU cache for on-demand
decompression during rendering
- `GfxRenderer` transparently routes bitmap access through
`getGlyphBitmap()` (compressed or direct flash)
- Uses `uzlib` for decompression with minimal heap overhead.
- 48 reader fonts (Bookerly, NotoSans 12-18pt, OpenDyslexic) regenerated
with compression; 5 UI fonts unchanged
- Round-trip verification script (`verify_compression.py`) runs as part
of font generation
## Additional Context

## Flash & RAM

| | baseline | font-compression | Difference |
|--|--------|-----------------|------------|
| Flash (ELF) | 6,302,476 B (96.2%) | 4,365,022 B (66.6%) | -1,937,454 B
(-30.7%) |
| firmware.bin | 6,468,192 B | 4,531,008 B | -1,937,184 B (-29.9%) |
| RAM | 101,700 B (31.0%) | 103,076 B (31.5%) | +1,376 B (+0.5%) |

## Script-Based Grouping (Cold Cache)

Comparison of uncompressed baseline vs script-based group compression
(4-slot LRU cache, cleared each page). Glyphs are grouped by Unicode
block (ASCII, Latin-1, Latin Extended-A, Combining Marks, Cyrillic,
General Punctuation, etc.) instead of sequential groups of 8.

### Render Time

| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Median** | 414.9 ms | 431.6 ms | +16.7 ms (+4.0%) |
| **Pages** | 37 | 37 | |

### Memory Usage

| | Baseline | Compressed (cold cache) | Difference |
|---|---|---|---|
| **Heap free (median)** | 187.0 KB | 176.3 KB | -10.7 KB |
| **Heap free (min)** | 186.0 KB | 166.5 KB | -19.5 KB |
| **Largest block (median)** | 148.0 KB | 128.0 KB | -20.0 KB |
| **Largest block (min)** | 148.0 KB | 120.0 KB | -28.0 KB |

### Cache Effectiveness

| | Misses/page | Hit rate |
|---|---|---|
| **Compressed (cold cache)** | 2.1 | 99.85% |

------

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

  Did you use AI tools to help write this code? _**YES**_
Implementation was done by Claude Code (Opus 4.6) based on a plan
developed collaboratively. All generated font headers were verified with
an automated round-trip decompression test. The firmware was compiled
successfully but has not yet been tested on-device.

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants