Skip to content

Bug with comparing ManifestArrays using __eq__ which have fill_value=NaN #501

Closed

Description

@TomNicholas

I'm stuck on why this assert_identical on the datasets doesn't pass. It complains that all the coordinates are different, but every attribute I can think to check looks identical 😖

Originally posted by @TomNicholas in #349 (comment)

I found out what the problem is - two ManifestArrays with identical .metadata will compare not equal if they both have NaN for a fill_value. This is because the __eq__ check introspects deeper until it finds e.g. the np.float32(nan) type, but

In [4]: bool(np.float32('nan') == np.float32('nan'))
Out[4]: False

(See https://stackoverflow.com/a/10059796 for why numpy NaNs behave like this.)

The solution needs to be to actually check two Metadata classes are __eq__ with dedicated code, not just trusting the python dataclasses' automatically-generated __eq__ method to fi it.

I would ideally like to move that fix upstream into zarr-python.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions