Skip to content

Commit a429d16

Browse files
d-v-boelhammouchiOthman El Hammouchi
authored
fix(storage): open ZipStore archive on first use from every method and reopen it without truncating (#4450)
* fix: ensure `ZipStore` is open before acquiring lock * refactor: Cleanup open check in ZipStore * test: Add test to check that lock is present when required * refactor: Add return type to `_sync_ensure_open` * docs: Add release note * fix(storage): open ZipStore archive through one accessor and reopen in append mode Create the lock in `__init__` and route every archive access through `_zipfile()`, which opens the archive on first use. This covers `get`, `get_partial_values`, `set_if_not_exists` and `clear`, which still raised `AttributeError` on a never-opened store. After the first open, switch mode "w" or "x" to "a" so that `move()` and unpickling reopen the archive without truncating it or refusing to open it. Builds on the fix proposed by Othman El Hammouchi in #3593. Refs #3588 Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Othman El Hammouchi <[email protected]> * docs: rename changelog fragment to PR number Assisted-by: ClaudeCode:claude-opus-5-5 * test(storage): make ZipStore reopen tests exercise the reopen Replace the close-twice case, which never reopened the archive and passed on main, with a read after close(), which erased a "w" archive on main. Assert return values and final archive contents in the first-use test instead of only checking that nothing raises. Cover reopening a file-object-backed store after close(). List the reopen after close(), the file-object case, and the first-use thread race in the changelog fragment. Assisted-by: ClaudeCode:claude-opus-5-5 * fix(storage): hold the ZipStore lock for all of close() close() marked the store closed before taking the lock, so a thread that used the store in that gap reopened the archive in append mode before its central directory was written. Its handle then wrote a directory that listed only its own entries, and the earlier entries were lost. Assisted-by: ClaudeCode:claude-opus-5-5 * fix(storage): hold the ZipStore lock across move() and every open move() closed the store, moved the file, and reopened it without the lock, so a thread that used the store in between reopened the old path and the move then failed with "store is already open". Hold the lock for the whole move. _open() also opens under the lock, and unpickling opens through _zipfile(). clear() keeps opening a never-opened store before deleting the file, so a store in mode "x" refuses an existing file instead of deleting it. A test pins that behaviour. Assisted-by: ClaudeCode:claude-opus-5-5 * docs: note move() in the ZipStore changelog fragment Assisted-by: ClaudeCode:claude-opus-5-5 * fix(storage): accept mode "x" in ZipStoreAccessModeLiteral ZipStore documents and accepts mode "x" at runtime, but the type of its mode parameter left it out, so type-checked callers needed an ignore. Assisted-by: ClaudeCode:claude-opus-5-5 * fix(storage): refuse to reopen a ZipStore on a write-only file object Reopening after close() uses append mode, which reads the archive back. A write-only file object cannot be read, so zipfile started a new archive after the old one and the earlier entries were silently lost. Raise io.UnsupportedOperation instead. The first open is unchanged. The thread-race tests now join the writer with a timeout, so a deadlock fails the test instead of hanging the run. Assisted-by: ClaudeCode:claude-opus-5-5 * fix(storage): make ZipStore close() exception-safe and check first use under the lock close() now marks the store closed even when closing the archive raises, so the next use reopens the archive instead of failing on a closed handle. ZipStore overrides _ensure_open, whose base version checked _is_open outside the lock, so it could race a concurrent first use and fail with "store is already open". Reuse after close() now also requires a seekable file object, since append mode seeks. The docstring now says that "w" and "x" apply to the first open only. A new test holds the lock while another thread makes its first use, which also catches a _zipfile() that opens without the lock. Assisted-by: ClaudeCode:claude-opus-5-5 * fix(storage): refuse to reopen a ZipStore whose close() failed If closing the archive raises, its central directory may be missing, and reopening it in append mode makes zipfile start a new archive and silently drop the earlier entries. The store now records the failure and raises RuntimeError on its next use. list() now iterates a copy of the names without holding the lock, so other threads are not blocked while a caller iterates. move() opens a never-opened store before closing it, so mode "x" refuses an existing file instead of moving it. The mode docstring no longer says every reopen appends. The failed-close test replaces one that patched ZipFile.close with a closure over the same ZipFile. That cycle raised from ZipFile.__del__ during a later test's garbage collection, which the warnings-as-errors config reported as an error in an unrelated test. Assisted-by: ClaudeCode:claude-opus-5-5 * fix(storage): leave a ZipStore closed when clear() fails partway clear() closed the archive, removed the file, and opened a new one. If a later step raised, the store stayed marked open on a closed handle and every use failed. It now closes the store through close() and reopens with "w", which truncates, so a failure leaves the store closed and the next use tries the truncating open again. The file is no longer removed first. Assisted-by: ClaudeCode:claude-opus-5-5 * fix(storage): let clear() recover a ZipStore whose close() failed clear() replaces the archive, so it cannot drop the entries that the failed-close guard protects; it now resets the guard instead of raising. clear() removes the file again before creating the new archive, as main did, so links, permissions, and other open readers behave as before. __getstate__ copies the state under the lock, so a pickle cannot catch _sync_open between the "w" open and the switch to "a". A test covers unpickling state from before _was_opened and _close_failed existed. Assisted-by: ClaudeCode:claude-opus-5-5 * test(storage): close ZipStores over file objects when a reuse test fails If the reuse-after-close tests fail, the store is still open when its file object goes away, and ZipFile.__del__ then raises during a later test's garbage collection. The warnings-as-errors config reported that as an error in an unrelated test. Close the store in a finally block. Assisted-by: ClaudeCode:claude-opus-5-5 * fix(storage): open unpickled ZipStores lazily and refuse to reopen an unfinished archive Unpickling opened the archive right away. A copy of a store that was still writing, like those dask makes while building a graph, found no central directory and opened in append mode, which zipfile marks as modified; when the copy was garbage-collected after the original closed, it wrote an empty central directory at the end of the file and the archive read as empty. Unpickled stores now open on first use, and a store that has already written refuses to reopen a file with no central directory, raising zipfile.BadZipFile. clear() lifts the failed-close guard only after the old file is removed, and the docstring and error message no longer suggest clear() for stores it cannot clear. A test covers clear() on a mode "r" store with read_only=False. Assisted-by: ClaudeCode:claude-opus-5-5 * docs: describe the unpickling fix from main's side in the ZipStore fragment Assisted-by: ClaudeCode:claude-opus-5-5 * fix(storage): let ZipStore.clear() replace a damaged or missing archive clear() opened the store before removing the file, so a store whose file had become a non-zip hit the reopen guard, and one whose file was gone after a failed close() raised FileNotFoundError forever. It now opens first only for a store that was never opened, which is what keeps mode "x" from deleting an unclaimed file, and removes the file with missing_ok. The guard's message no longer blames another writer alone, and the changelog says which concurrent writers it detects. Tests cover clear() on a damaged or missing file, reopening after the file was deleted, and a first open with "a" on an empty file. Assisted-by: ClaudeCode:claude-opus-5-5 * fix(storage): narrow ZipStore.clear()'s first open to mode "x" and report OS errors on reopen clear() opened a never-opened store in every mode, but only "x" needs it, and for mode "r" with read_only=False over a missing or damaged file that open failed and left the store unusable. The reopen guard used zipfile.is_zipfile, which swallows OSError, so a directory or an unreadable file at the path was reported as an unfinished archive; it now opens the file itself and lets such errors surface. The close() race test now starts the concurrent write while the central directory is being written, so removing the lock from close() fails it. New cases cover set_if_not_exists on an existing key, an old pickle in mode "a" over an empty file, KeyboardInterrupt during close(), clear() on a mode "r" store whose file is missing, and a directory at the archive path. Assisted-by: ClaudeCode:claude-opus-5-5 * refactor(storage): drop a redundant condition from ZipStore.clear() Mode "x" is switched to "a" by the first open, so checking _was_opened alongside it changed nothing. Assisted-by: ClaudeCode:claude-opus-5-5 * docs: credit #3593 in the ZipStore changelog fragment Assisted-by: ClaudeCode:claude-opus-5-5 * docs(zipstore): shorten the changelog fragment and give the new tests docstrings Assisted-by: ClaudeCode:claude-fable-5.1 --------- Co-authored-by: Othman El Hammouchi <[email protected]> Co-authored-by: Othman El Hammouchi <[email protected]> Co-authored-by: Othman El Hammouchi <[email protected]>
1 parent 44e89e7 commit a429d16

3 files changed

Lines changed: 672 additions & 45 deletions

File tree

‎changes/4450.bugfix.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Fixed several ways `ZipStore` could lose data or fail. A store created with `mode="w"` or `mode="x"` now keeps its entries when it is used again after `close()`, `move()`, or unpickling, instead of truncating the archive. Every method now opens the archive on first use, under a lock, so `get`, `get_partial_values`, `set_if_not_exists`, and `clear` no longer raise `AttributeError` on a store that was never opened, and threads sharing a store no longer open the archive twice. An unpickled store now opens its archive on first use rather than when it is unpickled, so writing a dask array to a `ZipStore` no longer produces an unreadable file. Where reopening an archive would silently drop its entries, the store now raises instead: `RuntimeError` after a failed `close()`, `zipfile.BadZipFile` if the file is not a complete zip archive, and `io.UnsupportedOperation` for a file object that is not readable and seekable. The type of the `mode` parameter now includes `"x"`. Builds on [#3593](https://github.com/zarr-developers/zarr-python/pull/3593) by [@oelhammouchi](https://github.com/oelhammouchi).

‎src/zarr/storage/_zip.py‎

Lines changed: 134 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121
if TYPE_CHECKING:
2222
from collections.abc import AsyncIterator, Iterable
2323

24-
ZipStoreAccessModeLiteral = Literal["r", "w", "a"]
24+
ZipStoreAccessModeLiteral = Literal["r", "w", "a", "x"]
2525

2626

2727
class _RawReaderAdapter(io.RawIOBase):
@@ -72,6 +72,17 @@ def readinto(self, b: Any) -> int:
7272
return n
7373

7474

75+
def _is_unfinished_archive(path: Path) -> bool:
76+
"""Whether the file at `path` exists but is not a zip archive."""
77+
try:
78+
with path.open("rb") as f:
79+
# is_zipfile swallows OSError, so open the file here to let a
80+
# directory or a permission error surface as itself
81+
return not zipfile.is_zipfile(f)
82+
except FileNotFoundError:
83+
return False
84+
85+
7586
class ZipStore(Store):
7687
"""
7788
Store using a ZIP file.
@@ -85,10 +96,20 @@ class ZipStore(Store):
8596
can only be used for reading (`mode="r"`). The file object must stay
8697
open for the lifetime of the store, and operations that require a
8798
filesystem location (`clear`, `move`, pickling) are not supported.
99+
Using the store again after `close()` reopens the archive, which
100+
requires a file object that is readable and seekable; otherwise it
101+
raises `io.UnsupportedOperation`.
88102
mode : str, optional
89103
One of 'r' to read an existing file, 'w' to truncate and write a new
90104
file, 'a' to append to an existing file, or 'x' to exclusively create
91-
and write a new file.
105+
and write a new file. 'w' and 'x' apply to the first open only; the
106+
store reopens its archive with 'a' after `close()`, `move()`, or
107+
unpickling, so the entries it already wrote are kept. An unpickled
108+
store opens its archive on first use. If `close()` raises, the
109+
archive may be incomplete, and every later use that would reopen it
110+
raises `RuntimeError` instead; for a writable store backed
111+
by a path, `clear()` replaces the archive and makes the store usable
112+
again.
92113
compression : int, optional
93114
Compression method to use when writing to the archive.
94115
allowZip64 : bool, optional
@@ -162,24 +183,78 @@ def __init__(
162183
self._zmode = mode
163184
self.compression = compression
164185
self.allowZip64 = allowZip64
186+
self._lock = threading.RLock()
187+
self._was_opened = False
188+
self._close_failed = False
165189

166190
def _sync_open(self) -> None:
167191
if self._is_open:
168192
raise ValueError("store is already open")
169-
170-
self._lock = threading.RLock()
193+
if self._close_failed:
194+
# the central directory may be missing; appending to such a file
195+
# makes zipfile start a new archive and drop the earlier entries
196+
raise RuntimeError(
197+
f"closing the archive of {self!r} failed, so it may be incomplete; "
198+
"the store will not reopen it"
199+
)
200+
if (
201+
self.path is not None
202+
and self._was_opened
203+
and self._zmode == "a"
204+
and _is_unfinished_archive(self.path)
205+
):
206+
# the file lacks a central directory, e.g. because another copy of
207+
# this store still has it open for writing; appending would start
208+
# a new archive and drop the entries already written
209+
raise zipfile.BadZipFile(
210+
f"{self!r} cannot reopen {self.path}: it is not a zip archive. It "
211+
"may be incomplete, for example because another store still has it "
212+
"open for writing."
213+
)
214+
if (
215+
self.path is None
216+
and self._fileobj is not None
217+
and self._was_opened
218+
and not (self._fileobj.readable() and self._fileobj.seekable())
219+
):
220+
# reopening appends, which needs to read the archive back; on a
221+
# write-only file zipfile would start a new archive and drop the
222+
# earlier entries
223+
raise io.UnsupportedOperation(
224+
"a ZipStore backed by a file object that is not readable and "
225+
"seekable cannot be used again after close(), because the "
226+
"archive it wrote cannot be read back"
227+
)
171228

172229
self._zf = zipfile.ZipFile(
173230
self.path if self.path is not None else self._fileobj, # type: ignore[arg-type]
174231
mode=self._zmode,
175232
compression=self.compression,
176233
allowZip64=self.allowZip64,
177234
)
235+
# "w" truncates and "x" refuses an existing file. Both apply only to the
236+
# first open: reopening after close(), move(), or unpickling must keep
237+
# the entries already written.
238+
if self._zmode in ("w", "x"):
239+
self._zmode = "a"
178240

241+
self._was_opened = True
179242
self._is_open = True
180243

244+
def _zipfile(self) -> zipfile.ZipFile:
245+
"""Return the archive, opening it on first use."""
246+
with self._lock:
247+
if not self._is_open:
248+
self._sync_open()
249+
return self._zf
250+
181251
async def _open(self) -> None:
182-
self._sync_open()
252+
with self._lock:
253+
self._sync_open()
254+
255+
async def _ensure_open(self) -> None:
256+
# the base class checks _is_open outside the lock
257+
self._zipfile()
183258

184259
def __getstate__(self) -> dict[str, Any]:
185260
if self.path is None:
@@ -189,24 +264,39 @@ def __getstate__(self) -> dict[str, Any]:
189264
"cannot pickle a ZipStore backed by a file-like object; "
190265
"construct the store from a path instead"
191266
)
192-
# We need a copy to not modify the state of the original store
193-
state = self.__dict__.copy()
267+
# We need a copy to not modify the state of the original store. The
268+
# lock keeps it from catching _sync_open between opening with "w" and
269+
# switching to "a", which would make unpickling truncate the archive.
270+
with self._lock:
271+
state = self.__dict__.copy()
194272
for attr in ["_zf", "_lock"]:
195273
state.pop(attr, None)
196274
return state
197275

198276
def __setstate__(self, state: dict[str, Any]) -> None:
199277
self.__dict__ = state
278+
self.__dict__.setdefault("_close_failed", False)
279+
self.__dict__.setdefault("_was_opened", False)
280+
self._lock = threading.RLock()
281+
# open on first use: a copy that is never used, like those dask makes
282+
# while building a graph, must not open the file, because zipfile
283+
# writes a central directory when it closes an archive it could not read
200284
self._is_open = False
201-
self._sync_open()
202285

203286
def close(self) -> None:
204287
# docstring inherited
205-
if not self._is_open:
206-
return
207-
super().close()
288+
# hold the lock until the archive is closed: a thread that reopened it
289+
# before its central directory was written would lose the entries
208290
with self._lock:
209-
self._zf.close()
291+
if not self._is_open:
292+
return
293+
try:
294+
self._zf.close()
295+
except BaseException:
296+
self._close_failed = True
297+
raise
298+
finally:
299+
super().close()
210300

211301
async def clear(self) -> None:
212302
# docstring inherited
@@ -216,11 +306,20 @@ async def clear(self) -> None:
216306
raise NotImplementedError(
217307
"clear() is not supported for a ZipStore backed by a file-like object"
218308
)
219-
self._zf.close()
220-
os.remove(self.path)
221-
self._zf = zipfile.ZipFile(
222-
self.path, mode="w", compression=self.compression, allowZip64=self.allowZip64
223-
)
309+
if self._zmode == "x":
310+
# "x" lasts only until the first open; opening now keeps it from
311+
# deleting a file this store never claimed
312+
self._zipfile()
313+
self.close()
314+
# the file may be damaged or gone; clear() replaces it either way
315+
self.path.unlink(missing_ok=True)
316+
# replacing the file cannot drop entries, so clear() is the one way
317+
# to recover a store whose close() failed
318+
self._close_failed = False
319+
# if this open fails the store stays closed, and the next use
320+
# creates the archive again
321+
self._zmode = "w"
322+
self._sync_open()
224323

225324
def __str__(self) -> str:
226325
if self.path is None:
@@ -243,11 +342,9 @@ def _get(
243342
prototype: BufferPrototype,
244343
byte_range: ByteRequest | None = None,
245344
) -> Buffer | None:
246-
if not self._is_open:
247-
self._sync_open()
248345
# docstring inherited
249346
try:
250-
with self._zf.open(key) as f: # will raise KeyError
347+
with self._zipfile().open(key) as f: # will raise KeyError
251348
if byte_range is None:
252349
return prototype.buffer.from_bytes(f.read())
253350
elif isinstance(byte_range, RangeByteRequest):
@@ -288,8 +385,6 @@ async def get_partial_values(
288385
return out
289386

290387
def _set(self, key: str, value: Buffer) -> None:
291-
if not self._is_open:
292-
self._sync_open()
293388
# generally, this should be called inside a lock
294389
keyinfo = zipfile.ZipInfo(filename=key, date_time=time.localtime(time.time())[:6])
295390
keyinfo.compress_type = self.compression
@@ -298,13 +393,11 @@ def _set(self, key: str, value: Buffer) -> None:
298393
keyinfo.external_attr |= 0x10 # MS-DOS directory flag
299394
else:
300395
keyinfo.external_attr = 0o644 << 16 # ?rw-r--r--
301-
self._zf.writestr(keyinfo, value.to_bytes())
396+
self._zipfile().writestr(keyinfo, value.to_bytes())
302397

303398
async def set(self, key: str, value: Buffer) -> None:
304399
# docstring inherited
305400
self._check_writable()
306-
if not self._is_open:
307-
self._sync_open()
308401
if not isinstance(value, Buffer):
309402
raise TypeError(
310403
f"ZipStore.set(): `value` must be a Buffer instance. Got an instance of {type(value)} instead."
@@ -315,7 +408,7 @@ async def set(self, key: str, value: Buffer) -> None:
315408
async def set_if_not_exists(self, key: str, value: Buffer) -> None:
316409
self._check_writable()
317410
with self._lock:
318-
members = self._zf.namelist()
411+
members = self._zipfile().namelist()
319412
if key not in members:
320413
self._set(key, value)
321414

@@ -337,23 +430,20 @@ async def delete(self, key: str) -> None:
337430

338431
async def exists(self, key: str) -> bool:
339432
# docstring inherited
340-
if not self._is_open:
341-
self._sync_open()
342433
with self._lock:
343434
try:
344-
self._zf.getinfo(key)
435+
self._zipfile().getinfo(key)
345436
except KeyError:
346437
return False
347438
else:
348439
return True
349440

350441
async def list(self) -> AsyncIterator[str]:
351442
# docstring inherited
352-
if not self._is_open:
353-
self._sync_open()
354-
with self._lock:
355-
for key in self._zf.namelist():
356-
yield key
443+
# namelist() is a copy; holding the lock across yield would block
444+
# other threads for as long as the caller iterates
445+
for key in self._zipfile().namelist():
446+
yield key
357447

358448
async def list_prefix(self, prefix: str) -> AsyncIterator[str]:
359449
# docstring inherited
@@ -363,11 +453,9 @@ async def list_prefix(self, prefix: str) -> AsyncIterator[str]:
363453

364454
async def list_dir(self, prefix: str) -> AsyncIterator[str]:
365455
# docstring inherited
366-
if not self._is_open:
367-
self._sync_open()
368456
prefix = prefix.rstrip("/")
369457

370-
keys = self._zf.namelist()
458+
keys = self._zipfile().namelist()
371459
seen = set()
372460
if prefix == "":
373461
keys_unique = {k.split("/")[0] for k in keys}
@@ -393,8 +481,12 @@ async def move(self, path: Path | str) -> None:
393481
)
394482
if isinstance(path, str):
395483
path = Path(path)
396-
self.close()
397-
os.makedirs(path.parent, exist_ok=True)
398-
shutil.move(self.path, path)
399-
self.path = path
400-
await self._open()
484+
# hold the lock so that no thread reopens the old path mid-move
485+
with self._lock:
486+
# opening first keeps mode "x" from moving a file it may not claim
487+
self._zipfile()
488+
self.close()
489+
os.makedirs(path.parent, exist_ok=True)
490+
shutil.move(self.path, path)
491+
self.path = path
492+
self._sync_open()

0 commit comments

Comments
 (0)