Listing all optional requirements of Dask#3456
Conversation
Includes all requirements of Dask without version constraints to start. Would like to fill these in as well. This should help us better understand what Dask needs for a complete working environment.
|
cc @mrocklin |
|
Thanks for doing this! Can I suggest that we rename this something like |
|
Sure. Had also thought about |
|
I like requirements-optional because I think it is the most clear that users don't need to install them. I'm mostly optimizing here to reduce the number of issues we receive from confused users. |
| bcolz | ||
| h5py | ||
| matplotlib | ||
| moto |
There was a problem hiding this comment.
Do we want to include testing dependencies like moto?
There was a problem hiding this comment.
Debated it, but included them initially as that was easier. Wasn't obvious to me where to draw the line. Please feel free to advise.
| partd | ||
| pytest | ||
| requests | ||
| s3fs |
mrocklin
changed the title
| moto | ||
| pandas_datareader | ||
| partd | ||
| pytest |
There was a problem hiding this comment.
Also what about optional dependencies of other projects that users might run into, pytables for dask.dataframe (implicitly depended on from pandas)
There was a problem hiding this comment.
Personally don't want to go down this road. It's too hard to tell what those might be and this goes far afield fast.
There was a problem hiding this comment.
I agree that it becomes a slippery slope, however I also suspect that pytables is more likely to be an issue for users than is scipy.
If the goal is to avoid packaging issues then it probably makes sense to also raise issues throughout the community? It seems odd to be the only ones doing this.
There was a problem hiding this comment.
Sorry I disagree. As we are not using pytables directly, it should not be a dependency. If we are concerned about how dask affects pytables, we should consider testing against it. At that point it becomes a testing dependency and can be listed for that reason.
To put this more generally, the reasoning should not be users may want to use this with Dask so it should be a dependency. Instead the reasoning should be, we aim to be compatible with this useful library through our testing and signify this already present commitment by listing it as a testing dependency. IOW dependency comes after usage not before.
There was a problem hiding this comment.
If the goal is to avoid packaging issues then it probably makes sense to also raise issues throughout the community? It seems odd to be the only ones doing this.
This happens often through engagement with packaging ecosystems like conda-forge and Linux distros. We also encourage projects to package licenses, changelogs, and point out other issues in a libraries metadata or build systems. In many cases CMakeLists.txt have been written for many projects by Windows users'/maintainers' in conda-forge seeking Windows support in libraries that had none. IOW this sort of stuff does happen.
| @@ -0,0 +1,15 @@ | |||
| bcolz | |||
| h5py | |||
There was a problem hiding this comment.
Also needs graphviz here.
|
Closing as I don't plan to work on this at present. |
2 participants
Includes all requirements of Dask without version constraints to start. Would like to fill these in as well. This should help us better understand what Dask needs for a complete working environment.
Note: Some things like
setuptoolsare needed at build time (i.e. to installdask) orversioneerfor updating the vendored versioning code, but are not needed outside of that. So they have been stripped.