Skip to content

fix pickle tests: avoid multiple simultaneous active writers - #273

Closed
ryan-williams wants to merge 1 commit into
zarr-developers:masterfrom
ryan-williams:pickles
Closed

fix pickle tests: avoid multiple simultaneous active writers#273
ryan-williams wants to merge 1 commit into
zarr-developers:masterfrom
ryan-williams:pickles

Conversation

@ryan-williams

@ryan-williams ryan-williams commented Jul 6, 2018

Copy link
Copy Markdown
Member

Several test_pickle cases fail for me on OSX (10.13.5) and in a CentOS (7.5) VM; here's a full example.

A minimal repro:

import tempfile
from zarr.storage import DBMStore
import pickle
path = tempfile.mktemp()
store = DBMStore(path, flag='n')
store['foo'] = b'bar'
pickled = pickle.dumps(store)
store2 = pickle.loads(pickled)

Running this gives:

Traceback (most recent call last):
  File "test.py", line 8, in <module>
    store2 = pickle.loads(pickled)
  File "/Users/ryan/c/hdf5-experiments/zarr/zarr/storage.py", line 1449, in __setstate__
    write_lock=write_lock, **open_kws)
  File "/Users/ryan/c/hdf5-experiments/zarr/zarr/storage.py", line 1425, in __init__
    self.db = open(path, flag, mode, **open_kwargs)
  File "/Users/ryan/.pyenv/versions/3.6.5/lib/python3.6/dbm/__init__.py", line 94, in open
    return mod.open(file, flag, mode)
_gdbm.error: [Errno 35] Resource temporarily unavailable

It seems to be related to having multiple file-descriptors open with "write" capabilities to a given path.

This fixes them for me locally; I don't know why they don't appear on Travis.

@ryan-williams ryan-williams mentioned this pull request Jul 6, 2018
Closed
@ryan-williams

ryan-williams commented Jul 6, 2018

Copy link
Copy Markdown
Member Author

The coverage decrease seems spurious to me; please advise?

@alimanfoo alimanfoo mentioned this pull request Oct 19, 2018
Merged
5 tasks done
@alimanfoo

alimanfoo commented Oct 19, 2018

Copy link
Copy Markdown
Member

Thanks again @ryan-williams for this. I ended up manually merging some changes based on this PR into another PR (#308), so this should be resolved in master now.

@alimanfoo alimanfoo closed this Oct 19, 2018
@alimanfoo alimanfoo added this to the v2.3 milestone Oct 19, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

v2.3

Development

Successfully merging this pull request may close these issues.

2 participants

@ryan-williams @alimanfoo