Skip to content

Replace setup.py with pyproject.toml and implement optional dependencies#166

Merged
EwoutH merged 2 commits intomasterfrom
pyproject
Sep 19, 2022
Merged

Replace setup.py with pyproject.toml and implement optional dependencies#166
EwoutH merged 2 commits intomasterfrom
pyproject

Conversation

@EwoutH
Copy link
Copy Markdown
Collaborator

@EwoutH EwoutH commented Sep 3, 2022 •
edited
Loading

This PR does two things:

  • It migrates the project install commands and metadata from setup.py to pyproject.toml. It largely follows the convention from setuptools.
  • It moves the dependencies from requirements.txt to the pyproject.toml file, and thereby also defines a few sets of optional depenencies. Note that many dependencies that were defined in the requirements.txt are now optional.

This PR succeeds #151 and closes #130.

To-do:

  • Migrate to pyproject.toml:
    • Project and Metadata
    • Dynamic version
    • Dependencies and optional dependencies
    • Version
    • Packaging
  • Update CI to use appropriate install commands for each job
  • Update installation docs
  • Implement "all" extras which installs everything
  • Test installing "all" extra on Linux and macOS (does Pythonnet error?)

The installation.rst docs are also updated to reflect the new options for installing extras.

@EwoutH EwoutH added this to the 2.3.0 milestone Sep 3, 2022
@coveralls
Copy link
Copy Markdown

coveralls commented Sep 3, 2022 •
edited
Loading

Coverage Status

Coverage increased (+0.05%) to 80.051% when pulling 6ca0178 on pyproject into da65c52 on master.

@EwoutH
Copy link
Copy Markdown
Collaborator Author

EwoutH commented Sep 3, 2022

@quaquel One thing still needs to be implemented, and I need a bit of help from you on that. That is the packaing, or what files/directories to include in the package. Currently there is quite a large script for, but I'm sure we could simplify that.

Could you tell me which folders and files need to be exactly included in the package? Then I will take an initial jab at implementing that in the pyproject.toml.

@EwoutH
Copy link
Copy Markdown
Collaborator Author

EwoutH commented Sep 3, 2022

With a bit of reverse engineering I figured it out. :)

The reverse-wildcard trick is brand new and not even documented in Setuptools yet (I found it in a PR, and requested it to be documented in the Data Files Support docs).

@EwoutH EwoutH added the docs label Sep 3, 2022
@EwoutH EwoutH requested a review from quaquel September 3, 2022 18:33
@EwoutH EwoutH marked this pull request as ready for review September 3, 2022 18:33
@EwoutH
Copy link
Copy Markdown
Collaborator Author

EwoutH commented Sep 3, 2022 •
edited
Loading

@quaquel Ready for review! Please also check the updated installation instructions, and if the right dependencies are assigned to each of the extras.

(please do not merge yet, even if everything looks goods, I want to clean the commits up before merging)

@quaquel
Copy link
Copy Markdown
Owner

quaquel commented Sep 5, 2022 •
edited
Loading

You might have to walk me through all these changes in person. A couple of things

  1. In the past, I had bad experiences with the automated installation of dependencies. I have seen it brake quite a few anaconda installations. This might be from before pyproject.toml so it might no longer be a problem
  2. I am not sure whether we want to break down the dependencies to the degree you have done here. Yes, keeping a light installation can be useful in some cases, but it also creates a possible problem for users that might not fully understand all the configuration options that they have. Again, we can discuss the exact options in person.
  3. I would not call ["pysd"] vensim as is now done in the updated installation instructions. Vensim itself only requires the DSS version and is really a different connector from pysd. I suggest just calling ["pysd"] pysd.

@EwoutH
Copy link
Copy Markdown
Collaborator Author

EwoutH commented Sep 6, 2022

Thanks for reviewing! I will look into your points, let's walk through them Monday.

Can we do 16:00 by the way? I won't be able to make 15:30.

EwoutH
EwoutH commented Sep 12, 2022
View reviewed changes
@EwoutH
Copy link
Copy Markdown
Collaborator Author

EwoutH commented Sep 12, 2022

@quaquel Turns out both JPype and pythonnet crash on other OSes than Windows.

In the latest commit:

  • Added graph to recommended extra
  • Removed the connector dependencies from all extra
  • Updated installation.rst docs

On my side it's ready, if you agree, I will clean it up and merge!

@quaquel quaquel mentioned this pull request Sep 14, 2022
Merged
3 tasks
@EwoutH
Copy link
Copy Markdown
Collaborator Author

EwoutH commented Sep 19, 2022

@quaquel If you approve I can cleanup and merge, and modify #170 to work with the new setup.

@quaquel
Copy link
Copy Markdown
Owner

quaquel commented Sep 19, 2022

looks good so go ahead

@EwoutH
Replace setup.py with pyproject.toml and implement optional dependencies
c389300 
This commit does three things:
 - It migrates the project install commands and metadata from `setup.py` to `pyproject.toml`. It largely follows the convention from https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html.
 - It moves the dependencies from `requirements.txt` to the `pyproject.toml` file, and thereby also defines a few sets of optional depenencies. Note that many dependencies that were defined in the `requirements.txt` are now optional.
- It updates the CI to work with this new configuration.
@EwoutH
Update installation docs for pyproject.toml and optional dependencies
6ca0178 
Update the installation docs (installation.rst) to mention the pyproject.toml configuration and how to use the extras which install optional dependencies.
@EwoutH EwoutH merged commit 6ca0178 into master Sep 19, 2022
@EwoutH EwoutH mentioned this pull request Sep 20, 2022
Closed
@EwoutH EwoutH deleted the pyproject branch September 26, 2022 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@quaquel quaquel Awaiting requested review from quaquel

Assignees

No one assigned

Labels

ci docs maintenance packaging

Projects

None yet

Milestone

2.3.0

Development

Successfully merging this pull request may close these issues.

Install dependencies based on core, dev, doc or test use case

3 participants

@EwoutH @coveralls @quaquel