Skip to content

DOC: Unclear how to use Zarr3 ArrayBytesCodec #4159

Closed

Description

@JeppeKlitgaard

Describe the issue linked to the documentation

I got really stuck trying to figure out how to use imagecodecs's (relatively) new zarr3-compatible codecs, where I specifically wanted to use the JPEG-XL codec.

After cursorily reading the zarr and imagecodecs documentations a few times, I still got stuck and started backtracking through my code until I finally realised that because imagecodecs.zarr.Jpegxl is an ArrayBytesCodec, it should not be passed in as a compressor, but as a serializer. I think this could be covered better in the documentation and doc-strings of zarr3, and perhaps also in the error messages.

I am not confident enough in my zarr3 knowledge yet to submit a PR myself, but hopefully you can see where I went wrong in this little example:

import numpy as np
import zarr
import imagecodecs.zarr

imagecodecs.zarr.register_codecs()
compressor = imagecodecs.zarr.Jpegxl(level=5)
ds = np.zeros(shape=(64, 256, 256))
z = zarr.create_array(
    store=COMPRESSION_TESTS_DIR / "out.zarr",
    shape=ds.shape,
    dtype=ds.dtype,
    chunks=(16, 128, 128),
    compressors=compressor,
    overwrite=True,
    zarr_format=3,
)
z[:] = ds

Which gives:

...
TypeError: Expected a BytesBytesCodec. Got <class 'imagecodecs.zarr.Jpegxl'> instead.

I ran into this: #1943, so clearly this transition was on your mind!

Activity

  1. added
    help wantedIssue could use help from someone with familiarity on the topic
    documentationImprovements to the documentation
    on Jul 17, 2026
  2. d-v-b commented on Jul 20, 2026

    @d-v-b
    Contributor

    quoting myself from the related imagecodecs issue:

    totally agree that we need to improve the docs here.

    Part of the confusion comes from the desire to make the create_array signature work for zarr v2 and zarr v3 arrays. Zarr V2 defines 2 fields: filters (list of numcodecs codecs or none) and compressor (numcodecs codec or none). Note that an element of filters could also be a compressor.

    Zarr V3 on the other hand has a more elaborate model of chunk encoding, where there are 3 types of codecs that vary in their encode / decode signatures, but at the same time, Zarr V3 uses a simpler metadata structure: there is just one codecs field. To make things more complicated, Zarr v3 codecs are recursive: some codecs have their own codecs.

    So we tried to smooth over the v2 / v3 differences with the create_array signature in particular. As a result, it's not obvious how those parameters relate to zarr v3 metadata. Sorry about that, we should definitely explain this in more detail.

    Longer term, I would like to make some basic changes to how we model array metadata, and build on that to do a better job with array creation APIs. there are several other things about create_array that haven't aged well, and should be improved.

    In terms of improvements we should make, I think if if we expose a function that talks about "filters", "serializer", "compression", that function should emit error messages that use the same terms. So the error message should probably say "Expected an instance of BytesBytesCodec for the compressor parameter, but got Got <class 'imagecodecs.zarr.Jpegxl'> instead."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements to the documentationhelp wantedIssue could use help from someone with familiarity on the topic

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions