Skip to content

Commit 4a3e5fa

Browse files
authored
Merge branch 'main' into feat/zarr-metadata-json-writer-config
2 parents b837a66 + 27f034e commit 4a3e5fa

17 files changed

Lines changed: 478 additions & 176 deletions

File tree

‎changes/4458.feature.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Reading part of an uncompressed chunk now fetches only the bytes it needs. When `BytesCodec` is the only codec, a selection is read with a single byte range request covering the rows along the first axis that it touches, instead of the whole chunk. Chunks with a compressor or any other codec are read whole, as before.

‎changes/4521.removal.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
The following have been removed, having been deprecated since Zarr-Python 3.0.0:
2+
3+
- `zarr.tree`, `zarr.api.synchronous.tree` and `zarr.api.asynchronous.tree`. Use `Group.tree` / `AsyncGroup.tree` instead.
4+
- The `Array.compressor` and `AsyncArray.compressor` properties. Use `Array.compressors` instead (for Zarr format 2 arrays, `compressors[0]` is the old `compressor`, and an empty tuple means no compressor).

‎changes/4522.removal.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
The Zarr-Python 2 keyword arguments `synchronizer`, `chunk_store`, `cache_metadata`, `cache_attrs`, `object_codec`, `read_only` and `meta_array` accepted by `zarr.create`, `zarr.group` and `zarr.open_group` (and the functions that wrap them, such as `zarr.zeros`) have never had any effect in Zarr-Python 3. Passing one of them now emits a `ZarrDeprecationWarning` instead of a "not yet implemented" `ZarrRuntimeWarning`. They will be removed in a release after February 2027. Calls that do not pass these arguments are unaffected.

‎changes/4612.bugfix.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
The `ArrayV2Metadata.chunk_grid` and `zarr.storage.default_compressor` deprecation warnings now point at the calling code instead of zarr internals, so Python no longer hides them by default.

‎docs/contributing.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -479,7 +479,7 @@ If, after weighing these factors, a removal is not clearly worthwhile, prefer to
479479

480480
Once a deprecation is decided, it must follow these steps so that users get a consistent, actionable signal:
481481

482-
1. **Emit a warning at runtime.** Raise `zarr.errors.ZarrDeprecationWarning` for an API that will be *removed*. Use `zarr.errors.ZarrFutureWarning` instead when behavior will *change* rather than disappear (for example, a default value that will change). The distinction is about who needs to see the warning. Python hides `DeprecationWarning` by default except in `__main__` and under test runners, so a removal warning reaches the authors of the code that calls the deprecated API, whereas `FutureWarning` is shown to everyone, which a behavior change needs because it alters results without any change to the user's code. The [`typing_extensions.deprecated`](https://typing-extensions.readthedocs.io/en/latest/#deprecated) decorator is the preferred way to deprecate whole functions, methods, and classes. When calling `warnings.warn` directly, set `stacklevel` so the warning points at the caller's code rather than zarr internals.
482+
1. **Emit a warning at runtime.** Raise `zarr.errors.ZarrDeprecationWarning` for an API that will be *removed*. Use `zarr.errors.ZarrFutureWarning` instead when behavior will *change* rather than disappear (for example, a default value that will change). The distinction is about who needs to see the warning. Python hides `DeprecationWarning` by default except in `__main__` and under test runners, so a removal warning reaches the authors of the code that calls the deprecated API, whereas `FutureWarning` is shown to everyone, which a behavior change needs because it alters results without any change to the user's code. The [`typing_extensions.deprecated`](https://typing-extensions.readthedocs.io/en/latest/#deprecated) decorator is the preferred way to deprecate whole functions, methods, and classes. When calling `warnings.warn` directly, set `stacklevel` so the warning points at the caller's code rather than zarr internals. This matters because Python shows a `DeprecationWarning` by default only when it points at `__main__`; one that points at zarr internals is seen only under test runners. A warning raised inside a coroutine cannot point at the caller when the synchronous API runs it, because `sync()` runs coroutines on a separate IO thread, so if the deprecation must reach users running scripts, emit it from the public synchronous function instead. Test the location, for example with `assert caught[0].filename == __file__`.
483483
2. **Write an actionable message.** The warning message must state the planned removal or change (a specific version if known, otherwise "a future release") and the migration path (i.e., what the user should do instead). A deprecation warning with no replacement guidance is incomplete.
484484
3. **Document it.** Add a "Deprecated" admonition to the docstring, stating the same removal target and migration path as the warning, so the deprecation appears in the rendered API reference, and update any affected user-guide prose.
485485
4. **Add a changelog entry.** Add a `removal` changelog fragment in `changes/` (for example, `changes/1234.removal.md`) so the deprecation is announced under "Deprecations and Removals" in the release notes. When the API is finally removed, add a second `removal` fragment for that release.

‎docs/user-guide/v3_migration.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -231,9 +231,9 @@ The following functions / methods have not been ported to Zarr-Python 3:
231231
- `zarr.copy_all` ([issue #2407](https://github.com/zarr-developers/zarr-python/issues/2407))
232232
- `zarr.copy_store` ([issue #2407](https://github.com/zarr-developers/zarr-python/issues/2407))
233233

234-
The following features (corresponding to function arguments to functions in
235-
`zarr`) have not been ported to Zarr-Python 3. Using these features
236-
will raise a warning or a `NotImplementedError`:
234+
The following function arguments from Zarr-Python 2 are accepted but ignored by
235+
Zarr-Python 3. Passing them emits a `ZarrDeprecationWarning`, and they will be
236+
removed in a release after February 2027:
237237

238238
- `cache_attrs`
239239
- `cache_metadata`

‎src/zarr/__init__.py‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,6 @@
3030
save,
3131
save_array,
3232
save_group,
33-
tree,
3433
zeros,
3534
zeros_like,
3635
)
@@ -179,7 +178,6 @@ def set_format(log_format: str) -> None:
179178
"save",
180179
"save_array",
181180
"save_group",
182-
"tree",
183181
"zeros",
184182
"zeros_like",
185183
]

‎src/zarr/api/asynchronous.py‎

Lines changed: 26 additions & 62 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,6 @@
77

88
import numpy as np
99
import numpy.typing as npt
10-
from typing_extensions import deprecated
1110

1211
from zarr.abc.store import Store
1312
from zarr.core.array import (
@@ -44,7 +43,6 @@
4443
GroupNotFoundError,
4544
NodeTypeValidationError,
4645
ZarrDeprecationWarning,
47-
ZarrRuntimeWarning,
4846
ZarrUserWarning,
4947
)
5048
from zarr.storage import StorePath
@@ -91,7 +89,6 @@
9189
"save",
9290
"save_array",
9391
"save_group",
94-
"tree",
9592
"zeros",
9693
"zeros_like",
9794
]
@@ -109,17 +106,23 @@ def _infer_overwrite(mode: AccessModeLiteral) -> bool:
109106
return mode in _OVERWRITE_MODES
110107

111108

112-
def _warn_unimplemented_kwargs(kwargs: dict[str, Any]) -> None:
109+
def _warn_ignored_kwargs(kwargs: dict[str, Any]) -> None:
113110
"""
114-
Emit a "not yet implemented" warning for each provided keyword argument that is not None.
111+
Emit a deprecation warning for each provided keyword argument that is not None.
112+
113+
These are Zarr-Python 2 keyword arguments that have no effect in Zarr-Python 3.
115114
116115
`kwargs` maps a keyword argument name to its supplied value. The `stacklevel` is chosen
117116
so the warning points at the caller of the public API function (the same location as an
118117
inline `warnings.warn(..., stacklevel=2)` would).
119118
"""
120119
for name, value in kwargs.items():
121120
if value is not None:
122-
warnings.warn(f"{name} is not yet implemented", ZarrRuntimeWarning, stacklevel=3)
121+
warnings.warn(
122+
f"{name} is ignored, and will be removed in a release after February 2027",
123+
ZarrDeprecationWarning,
124+
stacklevel=3,
125+
)
123126

124127

125128
def _get_shape_chunks(a: ArrayLike | Any) -> tuple[tuple[int, ...] | None, tuple[int, ...] | None]:
@@ -632,31 +635,6 @@ async def save_group(
632635
await asyncio.gather(*aws)
633636

634637

635-
@deprecated("Use AsyncGroup.tree instead.", category=ZarrDeprecationWarning)
636-
async def tree(grp: AsyncGroup, expand: bool | None = None, level: int | None = None) -> Any:
637-
"""Provide a rich display of the hierarchy.
638-
639-
!!! warning "Deprecated"
640-
`zarr.tree()` is deprecated since v3.0.0 and will be removed in a future release.
641-
Use `group.tree()` instead.
642-
643-
Parameters
644-
----------
645-
grp : Group
646-
Zarr or h5py group.
647-
expand : bool, optional
648-
Only relevant for HTML representation. If True, tree will be fully expanded.
649-
level : int, optional
650-
Maximum depth to descend into hierarchy.
651-
652-
Returns
653-
-------
654-
TreeRepr
655-
A pretty-printable object displaying the hierarchy.
656-
"""
657-
return await grp.tree(expand=expand, level=level)
658-
659-
660638
async def array(data: npt.ArrayLike | AnyArray, **kwargs: Any) -> AnyAsyncArray:
661639
"""Create an array filled with `data`.
662640
@@ -734,18 +712,15 @@ async def group(
734712
If True, delete any pre-existing data in `store` at `path` before
735713
creating the group.
736714
chunk_store : StoreLike or None, default=None
737-
Separate storage for chunks. Not implemented.
715+
Ignored. Deprecated and will be removed in a release after February 2027.
738716
cache_attrs : bool, optional
739-
If True (default), user attributes will be cached for attribute read
740-
operations. If False, user attributes are reloaded from the store prior
741-
to all attribute read operations.
717+
Ignored. Deprecated and will be removed in a release after February 2027.
742718
synchronizer : object, optional
743-
Array synchronizer.
719+
Ignored. Deprecated and will be removed in a release after February 2027.
744720
path : str, optional
745721
Group path within store.
746722
meta_array : array-like, optional
747-
An array instance to use for determining arrays to create and return
748-
to users. Use `numpy.empty(())` by default.
723+
Ignored. Deprecated and will be removed in a release after February 2027.
749724
zarr_format : {2, 3, None}, optional
750725
The zarr format to use when saving.
751726
storage_options : dict
@@ -858,23 +833,18 @@ async def open_group(
858833
(fail if exists). On a store that cannot delete keys, 'w' raises an
859834
error instead of replacing an existing node.
860835
cache_attrs : bool, optional
861-
If True (default), user attributes will be cached for attribute read
862-
operations. If False, user attributes are reloaded from the store prior
863-
to all attribute read operations.
836+
Ignored. Deprecated and will be removed in a release after February 2027.
864837
synchronizer : object, optional
865-
Array synchronizer.
838+
Ignored. Deprecated and will be removed in a release after February 2027.
866839
path : str, optional
867840
Group path within store.
868841
chunk_store : StoreLike or None, default=None
869-
Separate storage for chunks. See the
870-
[storage documentation in the user guide][user-guide-store-like]
871-
for a description of all valid StoreLike values.
842+
Ignored. Deprecated and will be removed in a release after February 2027.
872843
storage_options : dict
873844
If using an fsspec URL to create the store, these will be passed to
874845
the backend implementation. Ignored otherwise.
875846
meta_array : array-like, optional
876-
An array instance to use for determining arrays to create and return
877-
to users. Use `numpy.empty(())` by default.
847+
Ignored. Deprecated and will be removed in a release after February 2027.
878848
attributes : dict
879849
A dictionary of JSON-serializable values with user-defined attributes.
880850
use_consolidated : bool or str, default None
@@ -900,7 +870,7 @@ async def open_group(
900870
The new group.
901871
"""
902872

903-
_warn_unimplemented_kwargs(
873+
_warn_ignored_kwargs(
904874
{
905875
"cache_attrs": cache_attrs,
906876
"synchronizer": synchronizer,
@@ -1005,15 +975,14 @@ async def create(
1005975
[storage documentation in the user guide][user-guide-store-like]
1006976
for a description of all valid StoreLike values.
1007977
synchronizer : object, optional
1008-
Array synchronizer.
978+
Ignored. Deprecated and will be removed in a release after February 2027.
1009979
overwrite : bool, optional
1010980
If True, delete all pre-existing data in `store` at `path` before
1011981
creating the array.
1012982
path : str, optional
1013983
Path under which array is stored.
1014984
chunk_store : StoreLike or None, default=None
1015-
Separate storage for chunks. If not provided, `store` will be used
1016-
for storage of both chunks and metadata.
985+
Ignored. Deprecated and will be removed in a release after February 2027.
1017986
filters : Iterable[Codec] | Literal["auto"], optional
1018987
Iterable of filters to apply to each chunk of the array, in order, before serializing that
1019988
chunk to bytes.
@@ -1034,18 +1003,13 @@ async def create(
10341003
10351004
To create an array with no filters, provide an empty iterable or the value `None`.
10361005
cache_metadata : bool, optional
1037-
If True, array configuration metadata will be cached for the
1038-
lifetime of the object. If False, array metadata will be reloaded
1039-
prior to all data access and modification operations (may incur
1040-
overhead depending on storage and data access pattern).
1006+
Ignored. Deprecated and will be removed in a release after February 2027.
10411007
cache_attrs : bool, optional
1042-
If True (default), user attributes will be cached for attribute read
1043-
operations. If False, user attributes are reloaded from the store prior
1044-
to all attribute read operations.
1008+
Ignored. Deprecated and will be removed in a release after February 2027.
10451009
read_only : bool, optional
1046-
True if array should be protected against modification.
1010+
Ignored. Deprecated and will be removed in a release after February 2027.
10471011
object_codec : Codec, optional
1048-
A codec to encode object arrays, only needed if dtype=object.
1012+
Ignored. Deprecated and will be removed in a release after February 2027.
10491013
dimension_separator : {'.', '/'}, optional
10501014
Separator placed between the dimensions of a chunk.
10511015
Zarr format 2 only. Zarr format 3 arrays should use `chunk_key_encoding` instead.
@@ -1062,7 +1026,7 @@ async def create(
10621026
which instructs Zarr to choose the default Zarr format value defined in the
10631027
runtime configuration.
10641028
meta_array : array-like, optional
1065-
Not implemented.
1029+
Ignored. Deprecated and will be removed in a release after February 2027.
10661030
attributes : dict[str, JSON], optional
10671031
A dictionary of user attributes to store with the array.
10681032
chunk_shape : ChunksLike, optional
@@ -1128,7 +1092,7 @@ async def create(
11281092
if zarr_format is None:
11291093
zarr_format = _default_zarr_format()
11301094

1131-
_warn_unimplemented_kwargs(
1095+
_warn_ignored_kwargs(
11321096
{
11331097
"synchronizer": synchronizer,
11341098
"chunk_store": chunk_store,

0 commit comments

Comments
 (0)