Zarr v3: TypeError on float fill_value "0.0" for uint8 arrays—compatibility with legacy datasets #3447

Activity

1. thewtex

   added

   bugPotential issues with the zarr-python libraryPotential issues with the zarr-python library

   on Sep 11, 2025
2. d-v-b commented on Sep 11, 2025

   

   d-v-b

   on Sep 11, 2025

   Contributor

   More actions

   Should zarr-python accept 0.0 as a compatible fill_value for integer arrays and transparently cast it?

   Yes, we should do this. IMO there's no need for a warning or error here.

   👍React with 👍2Reacted by Matt McCormick and Christian Tischer
3. d-v-b commented on Sep 12, 2025

   

   d-v-b

   on Sep 12, 2025

   Last edited by d-v-b

   Contributor

   More actions

   #3448 will fix this, until then here's a workaround:

   # /// script
   # requires-python = ">=3.11"
   # dependencies = [
   #   "requests",
   #   "aiohttp",
   #   "fsspec",
   #   "zarr@git+https://github.com/zarr-developers/zarr-python.git@main",
   # ]
   # ///
   #
   # This script automatically imports the development branch of zarr to check for issues
   
   from typing import Any
   from numpy import unsignedinteger
   from numpy._typing._nbit_base import _8Bit
   import zarr
   from zarr.core.common import ZarrFormat
   from zarr.dtype import UInt8
   from zarr.dtype import data_type_registry
   
   class FixedUInt8(UInt8):
       def from_json_scalar(self, data: Any, *, zarr_format: ZarrFormat) -> Any:
           if isinstance(data, float):
              return super().from_json_scalar(int(data), zarr_format=zarr_format)
           return super().from_json_scalar(data, zarr_format=zarr_format)
   
   data_type_registry.register('uint8', FixedUInt8)
   
   url = "https://s3.embl.de/i2k-2020/platy-raw.ome.zarr"
   component = "s0"  # Try s0, but other resolutions likely affected
   
   # Attempt to open remote array
   z = zarr.open_array(store=url, path=component)
   print(z)
   
   zarr.print_debug_info()

   The logic here is that you override the json deseralization logic to convert floats to ints, then register that data type under its zarr v3 name (e.g., "uint8"). Repeat as needed for the other data types.
4. d-v-b commented on Sep 12, 2025

   

   d-v-b

   on Sep 12, 2025

   Contributor

   More actions

   would it be helpful to include this kind of example in our documentation? I think overriding data type JSON deserialization behavior is very much in scope for our data type registry design.
5. d-v-b

   closed this as completedin #3448on Sep 12, 2025
6. ianhi commented on Sep 23, 2025

   

   ianhi

   on Sep 23, 2025

   Last edited by ianhi

   Contributor

   More actions

   would it be helpful to include this kind of example in our documentation? I think overriding data type JSON deserialization behavior is very much in scope for our data type registry design.

   Yes! and ideally the error messages related to this type of issue would point at those docs
7. thewtex commented on Sep 23, 2025

   

   thewtex

   on Sep 23, 2025

   Author

   More actions

   @d-v-b thank you for the fix!