Include images from raw pixmaps and more by frozolotl · Pull Request #5632 · typst/typst

frozolotl · 2024-12-28T02:21:27Z

This PR adds a new API to include images from pixmaps:

#image.decode(
  (
    data: bytes(..),
    pixel-width: 32,
    pixel-height: 32,
  ),
  format: "rgb8",
  width: 1cm,
)

Excuses

Originally I wanted to do that in this PR, but I think it's better if I do individual PRs for the remaining changes.
I have promised this PR for far too long without publicizing these changes and the knowledge of still having outstanding work here hindered me from working on other features, even when I wanted to.
Furthermore, I needed to rebase (with conflicts) this PR twice and rewrote it once when the rebase was too painful. That wasn't a lot of fun.
It's time to stop all that.

Reasoning

This allows plugins to generate images without requiring them to export to PNG or another supported format.
Typst can then encode them in the way best fitting to the output format.

Such a thing can be useful when writing plugins that decode additional image formats, or for generating graphics where the Typst primitives would not be ideal (performance or looks).

Changes

Add pixmap API (as shown above).
- Support formats:
  - "rgb8": The typical RGB channels, each eight bits in size.
  - "rgba8": Same, but with alpha.
  - "luma8": Grayscale images
  - "lumaa8": …with alpha.
  - There are a number of other formats that would be useful, but I have refrained from adding them for now. This can be done in future changesets.
- ICC profile can be specified by the user.
Add option for image scaling algorithm.
- auto: Let Typst or the viewer decide automatically.
- "smooth": Use a "smooth" image scaling algorithm, like bilinear or cubic interpolation.
- "pixelated": Display the image pixelated, scaled using nearest neighbor.
Fix issue where images were scaled a pixel off in PNG export.

API

The API proposed in this PR is admittedly not great. It works, but it doesn't feel very good and discoverability isn't that good either.

There is an obvious alternative, one I actually implemented in a previous revision of this feature.

#image.from-pixmap(
  bytes(..),
  pixel-width: 32,
  pixel-height: 32,
  format: "rgba8",
)

It doesn't fit perfectly well into Typst as it is right now either, though.
First, pixel-width and pixel-height are required parameters. Typst currently doesn't do named required parameters.
This can be solved by making them named, taking an Option<u32>, but throwing an error on the None case. Not ideal.
I don't like making these parameters unnamed that much either, though.

The other problem is that this would require defining another duplicate of all image fields. These by now are quite a number¹.
This increases the maintenance burden where a single change in documentation needs to be replicated an additional two times.
It could in theory be solved using macros, but that's not that great either.

With the prospective changes to resources, the image.decode method will be removed, decreasing the amount of duplication. Whether that increases or decreases the appeal of either of the two APIs, I am not sure.

Future Work

This PR does not add an option to compress the images lossily. This still makes it badly suited for plugins that try to support additional image formats. Users who include a well-compressed photo could see significant output sizes after conversion.
This shouldn't be very difficult to implement; it mostly depends on the API design.

For the plotting use-case, it might be useful to generate pixmaps from color arrays. There isn't any blocker here.

I already mentioned it, but there are a number of formats that could be useful:

more channel sizes (bitmaps, 2 bits, 4 bits, 16 bits, 32 bits, etc.)
floats instead of integers
other color spaces (e.g. XYZ)
different channel orders (e.g. BGR, ABGR)

And then there is also room for optimization².

Closes #2575 #1400.

Closes typst#2575.

Closes typst#1400.

It seems we used the wrong method in the past, causing images to not be the expected size. The image-fit test shows this well, I think.

laurmaedje · 2025-01-30T19:01:38Z

I refactored a whole bunch and changed the API to this style:

#image(
  bytes(range(16).map(x => x * 16)),
  format: (
    encoding: "luma8",
    width: 4,
    height: 4,
  ),
  width: 1cm,
  icc: "sRGB-v4.icc",
)

The source argument is kept like on main, the pixel width and height moved into the format, and icc is its own argument. This eliminates the two error paths for mismatch between source and format and allowed me to unify the raster and pixmap reprs again. Specifying the icc profile separately means that one can also use this argument to override the ICC profile of a normal raster image.

This would now be ready to merge from my end.

laurmaedje · 2025-01-31T10:01:41Z

Thank you for your work on this! :)

git download method fixed warinings documentation cli updater adaptation and clippy fixes enhanced documentation add git downloader default impl migrating from git2 to gitoxide crate for git downloads Add support for `c2sc` OpenType feature in `smallcaps` (typst#5655) Just add MathText SyntaxKind Basic SymbolElem addition Use SymbolElem in more places and add `char` cast for content Add SymbolElem to realization Update math TextElem layout to separate out SymbolElem Handle boxes and blocks a bit better in HTML export (typst#5744) Co-authored-by: Martin Haug <[email protected]> Tweak HTML pretty printing (typst#5745) Semantic paragraphs (typst#5746) Fix space collapsing for explicit paragraphs (typst#5749) Support first-line-indent for every paragraph (typst#5768) Fixed typo in the new outline docs (typst#5772) Resolve bound name of bare import statically (typst#5773) Fix typo in scripting.md (typst#5783) Modular, multi-threaded, transitioning plugins (typst#5779) Include images from raw pixmaps and more (typst#5632) Co-authored-by: PgBiel <[email protected]> Co-authored-by: Laurenz <[email protected]> Change type repr to short name (typst#5788) Disable cjk_latin_spacing in raw by default (typst#5753) Change the default math class of U+22A5 ⊥ UP TACK to Normal (typst#5714) Revert adding `flatten-text` to `image` (typst#5789) Refactor `Scope` (typst#5797) Enable HTML feature in docs generator (typst#5800) Scope deprecations (typst#5798) Fix typo in page documentation (typst#5804) Bump openssl from 0.10.66 to 0.10.70 (typst#5802) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Bump codex to 0.1.0 (typst#5805) Bump dependencies (typst#5808) Fix small copy-paste oversight (typst#5811) Bump more dependencies (typst#5813) Export target docs (typst#5812) Co-authored-by: Martin Haug <[email protected]> 0.13 changelog (typst#5801) Release Candidate 1 Autocomplete content methods (typst#5822) Documentation fixes and improvements (typst#5816) Fix docs outline for nested definitions (typst#5823) Document removals in changelog (typst#5827) Fix unnecessary import rename warning (typst#5828) Don't crash on image with zero DPI (typst#5835) Add warning for `pdf.embed` elem used with HTML (typst#5829) Add smart quotes for Bulgarian (typst#5807) Respect `par` constructor arguments (typst#5842) Bump `typst-assets` Fix autocomplete and jumps in math (typst#5849) Update documentation for `float.{to-bits, from-bits}` (typst#5836) `Gradient::repeat`: Fix floating-point error in stop calculation (typst#5837) Lazy parsing of the package index (typst#5851) Remove Linux Libertine warning (typst#5876) Bring back type/str compatibility for 0.13, with warnings and hints (typst#5877) More robust SVG auto-detection (typst#5878) HTML export: Use `<code>` for inline `RawElem` (typst#5884) --make-deps fixes (typst#5873) Update changelog (typst#5894) Version bump Fix HTML export of table with gutter (typst#5920) Fix comparison of `Func` and `NativeFuncData` (typst#5943) HTML export: fix elem counting on classify_output (typst#5910) Co-authored-by: Laurenz <[email protected]> Fix introspection of HTML root sibling metadata (typst#5953) Fix high CPU usage due to inotify watch triggering itself (typst#5905) Co-authored-by: Laurenz <[email protected]> Fix false positive for type/str comparison warning (typst#5957) Fix paper name in page setup guide (typst#5956) Fix curve with multiple non-closed components. (typst#5963) Fix docs example with type/string comparison (typst#5987) Correct typo (typst#5971) Make `array.chunks` example more readable (typst#5975) Hotfix for labels on symbols (typst#6015) Replace `par` function call in tutorial (typst#6023) Mention that `sym.ohm` was removed in the 0.13.0 changelog (typst#6017) Co-authored-by: Laurenz <[email protected]> Mark breaking symbol changes as breaking in 0.13.0 changelog (typst#6024) 0.13.1 changelog (typst#6025) Version bump dep min 1.81 use 1.81 in ci

frozolotl added 5 commits December 28, 2024 03:33

feat: add pixmap support

259a029

Closes typst#2575.

refactor: merge Image::new and Image::with_fonts

4993bae

feat: allow user to choose how images should be scaled

9789fdb

Closes typst#1400.

perf: allocate a little less often

60045fa

fix: use resize_exact instead of resize

8453fa7

It seems we used the wrong method in the past, causing images to not be the expected size. The image-fit test shows this well, I think.

frozolotl force-pushed the pixmap branch from 4247c90 to 8453fa7 Compare December 28, 2024 02:33

refactor(pdf): rename icc_profile to be more clear

1cf4123

frozolotl mentioned this pull request Dec 29, 2024

Add support for higher bit depths, do not re-encode JPEGs #5637

Closed

PgBiel and others added 13 commits January 30, 2025 02:37

merge pixmap and main

477e1b2

attempt 2: use datasource

283fcbb

Refactor

3862102

Fix docs link

9362c0e

Refactor a bit more

cd2a61c

A bit of docs

574952f

Move metadata into format, unifying pixmap and raster image

1e5b82f

Refactor a bit

19e719f

Fix ImageElem::decode

e00508f

Add icc argument

b68971d

Plain image constructors

776c6d1

Two small docs improvements

1e641be

Also hash ICC profile

8f884f9

Luma is not brightness

b6e8864

laurmaedje enabled auto-merge January 31, 2025 09:54

laurmaedje added this pull request to the merge queue Jan 31, 2025

Merged via the queue into typst:main with commit 3eb6e87 Jan 31, 2025
6 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Include images from raw pixmaps and more#5632

Include images from raw pixmaps and more#5632
laurmaedje merged 20 commits intotypst:mainfrom
frozolotl:pixmap

frozolotl commented Dec 28, 2024 •

edited

Loading

Uh oh!

laurmaedje commented Jan 30, 2025

Uh oh!

Uh oh!

laurmaedje commented Jan 31, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

frozolotl commented Dec 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Excuses

Reasoning

Changes

API

Future Work

Footnotes

Uh oh!

laurmaedje commented Jan 30, 2025

Uh oh!

Uh oh!

laurmaedje commented Jan 31, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

frozolotl commented Dec 28, 2024 •

edited

Loading