Repository navigation
[Unexpected Behaviour] open_group raises AssertionError if array w/o group exists #3114
Description
Zarr version
3.0.8
Numcodecs version
0.15.1
Python Version
3.11.11
Operating System
Linux, Dockerfile based on python:3.11-slim
Installation
poetry
Description
I think there is some unexpected behavior that occurs if one
- creates an array in the root of a store
- attempts to open a group in the root of this store in
rorr+mode.
I would have expected some meaningful error, such as the zarr.errors.ContainsArrayError, or FileNotFoundError, but one just gets a very vague AssertionError.
I would like to avoid catching such a broad error in my code.
Below you can find a reproducible test code (tested with uv run) that highlights the expected and actually occuring behaviour.
Note on Stores:
I originally stumbled upon it when using the object store, and could directly reproduce it with the memory store, so it should be independent of the store.
Steps to reproduce
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "zarr@git+https://github.com/zarr-developers/zarr-python.git@main",
# "pytest>=8.3",
# ]
# ///
# This script automatically imports the development branch of zarr to check for issues
import pytest
import zarr.errors
import zarr.storage
def test_open_group_in_dir_with_array():
store = zarr.storage.MemoryStore()
# this passes
with pytest.raises(FileNotFoundError):
zarr.open_group(store, mode="r+")
# now create an array within root
zarr.create_array(store, shape=(2,), dtype=float)
# this somehow raises an assertion error ?
# I would expect zarr.errors.ContainsArrayError or FileNotFoundError.
# with pytest.raises(AssertionError):
with pytest.raises((zarr.errors.ContainsArrayError, FileNotFoundError)):
zarr.open_group(store, mode="r+")
def test_create_group_in_dir_with_array():
store = zarr.storage.MemoryStore()
# this passes
with pytest.raises(FileNotFoundError):
zarr.open_group(store, mode="r+")
# now create an array within root
zarr.create_array(store, shape=(2,), dtype=float)
with pytest.raises(zarr.errors.ContainsArrayError):
tmp = zarr.create_group(store)
if __name__ == "__main__":
pytest.main([__file__])Additional output
The test_open_group_in_dir_with_arrray fails with the following stacktrace:
==================================================================== FAILURES ====================================================================
_______________________________________________________ test_open_group_in_dir_with_array ________________________________________________________
def test_open_group_in_dir_with_array():
store = zarr.storage.MemoryStore()
# this passes
with pytest.raises(FileNotFoundError):
zarr.open_group(store, mode="r+")
# now create an array within root
zarr.create_array(store, shape=(2,), dtype=float)
# this somehow raises an assertion error ?
# I would expect zarr.errors.ContainsArrayError or FileNotFoundError.
# with pytest.raises(AssertionError):
with pytest.raises((zarr.errors.ContainsArrayError, FileNotFoundError)):
> zarr.open_group(store, mode="r+")
zarr_bug.py:28:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/vscode/.cache/uv/environments-v2/zarr-bug-4f65dcfdf7520214/lib/python3.11/site-packages/zarr/_compat.py:43: in inner_f
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
/home/vscode/.cache/uv/environments-v2/zarr-bug-4f65dcfdf7520214/lib/python3.11/site-packages/zarr/api/synchronous.py:529: in open_group
sync(
/home/vscode/.cache/uv/environments-v2/zarr-bug-4f65dcfdf7520214/lib/python3.11/site-packages/zarr/core/sync.py:163: in sync
raise return_result
/home/vscode/.cache/uv/environments-v2/zarr-bug-4f65dcfdf7520214/lib/python3.11/site-packages/zarr/core/sync.py:119: in _runner
return await coro
^^^^^^^^^^
/home/vscode/.cache/uv/environments-v2/zarr-bug-4f65dcfdf7520214/lib/python3.11/site-packages/zarr/api/asynchronous.py:828: in open_group
return await AsyncGroup.open(
/home/vscode/.cache/uv/environments-v2/zarr-bug-4f65dcfdf7520214/lib/python3.11/site-packages/zarr/core/group.py:583: in open
return cls._from_bytes_v3(
/home/vscode/.cache/uv/environments-v2/zarr-bug-4f65dcfdf7520214/lib/python3.11/site-packages/zarr/core/group.py:647: in _from_bytes_v3
return cls.from_dict(store_path, group_metadata)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/home/vscode/.cache/uv/environments-v2/zarr-bug-4f65dcfdf7520214/lib/python3.11/site-packages/zarr/core/group.py:656: in from_dict
metadata=GroupMetadata.from_dict(data),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cls = <class 'zarr.core.group.GroupMetadata'>
data = {'attributes': {}, 'chunk_grid': {'configuration': {'chunk_shape': [2]}, 'name': 'regular'}, 'chunk_key_encoding': {'c...ion': {'endian': 'little'}, 'name': 'bytes'}, {'configuration': {'checksum': False, 'level': 0}, 'name': 'zstd'}], ...}
@classmethod
def from_dict(cls, data: dict[str, Any]) -> GroupMetadata:
data = dict(data)
> assert data.pop("node_type", None) in ("group", None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E AssertionError
Activity
[-]Inconsistent & Unexpected Behaviour: open_group raises AssertionError if Array exists[/-][+][Unexpected Behaviour] open_group raises AssertionError if array w/o group exists[/+][-][Unexpected Behaviour] open_group raises AssertionError if array w/o group exists[/-][+][Unexpected Behaviour] `open_group` raises `AssertionError` if array w/o group exists[/+]d-v-b commentedon Jun 4, 2025
agreed that an assertion error here is not very helpful. I'll have a look at this today
TomNicholas commentedon Sep 9, 2025
I just ran into this - it's very ugly.
I'm surprised that all stores do not have a root group by definition??
This also makes it impossible to use
.treeon some stores:raises
AssertionError.AFAICT there is no way to get the root group, and no other way to call
.tree()other than on aGroupinstance.dstansby commentedon Sep 9, 2025
Isn't it possible (and reasonable) to have a store with just a single array and no groups, as in the code example above? In which case trying to open a group should error (but in a nicer way than an
AssertionError).TomNicholas commentedon Sep 9, 2025
I personally think that's a bad design choice, and that there should always be a root group. But I understand that that is a spec-level problem.
Separately I would still expect to be able to use
.treeon such a group-less store. But if.treeis only available onGroupand not all stores have aGroupthen.treecannot always be available. That's an example of why not having a root group is a bad design IMO - you forgo the useful property that methods on groups in the tree are available at all levels.d-v-b commentedon Sep 9, 2025
Not saying the root array concept is a good idea or not, but requiring a root group would complicate any API for directly creating an array without first defining a group, which is a pattern a lot of people use.
That being said, we can still define a
treemethod on arrays that does what you expect. thetreemethod on the group could even just be defined as calling thetreemethod on its members.