Skip to content

Handle environments where locks are not available - #83

Merged
alimanfoo merged 3 commits into
zarr-developers:masterfrom
mckinsel:mckinsel-fix-lambda-lock
Nov 6, 2018
Merged

alimanfoo merged 3 commits into
zarr-developers:masterfrom
mckinsel:mckinsel-fix-lambda-lock

Conversation

@mckinsel

Copy link
Copy Markdown
Contributor

Blosc want to acquire a lock, but locks aren't available in all environments. In particular, AWS Lambda lacks /dev/shm, so trying to use blosc will raise an OSError:

>>> import numcodecs.blosc
OSError: [Errno 38] Function not implemented

This changes handles missing locks more gracefully by just turning of threads.

@alimanfoo

alimanfoo commented Aug 29, 2018 via email

Copy link
Copy Markdown
Member

@tomwhite

tomwhite commented Oct 5, 2018

Copy link
Copy Markdown
Member

I hit the same problem on AWS Lambda, and needed to handle another case in numcodecs __init__.py file where the number of cores is queried and multiprocessing throws another OSError. I used this fix (in addition to Marcus' one in this PR): tomwhite@c54bc3d

@alimanfoo

Copy link
Copy Markdown
Member

Thanks @tomwhite for the feedback. This should be prioritised for the next release.

@alimanfoo alimanfoo added this to the 0.6.0 milestone Oct 8, 2018
@tomwhite

tomwhite commented Oct 8, 2018

Copy link
Copy Markdown
Member

@alimanfoo that would be great. Thanks.

@alimanfoo alimanfoo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, sorry for delay in reviewing.

@alimanfoo
alimanfoo merged commit d645bdd into zarr-developers:master Nov 6, 2018
@alimanfoo

Copy link
Copy Markdown
Member

Thanks @mckinsel.

@tomwhite tomwhite mentioned this pull request Nov 6, 2018
@tomwhite

tomwhite commented Nov 6, 2018

Copy link
Copy Markdown
Member

@alimanfoo thanks for merging this. Would you be able to take a look at the related change in #93 too please, as I needed that change to successfully run on AWS Lambda.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants