Spring cleaning for dependencies, Pt. 2: The Tree-Chopping by lbianchi-lbl · Pull Request #1133 · IDAES/idaes-pse

lbianchi-lbl · 2023-03-15T22:52:35Z

Resolves: #705

Motivation

Running pip install "." from this branch in a fresh environment results in the following being installed:

  Package         Version
--------------- ----------
certifi         2022.12.7
click           8.1.3
contourpy       1.0.7
cycler          0.11.0
fonttools       4.39.0
idaes-pse       2.0.0.dev3
kiwisolver      1.4.4
matplotlib      3.7.1
numpy           1.24.2
packaging       23.0
pandas          1.5.3
Pillow          9.4.0
pip             23.0.1
ply             3.11
Pyomo           6.5.0
pyparsing       3.0.9
python-dateutil 2.8.2
pytz            2022.7.1
scipy           1.10.1
setuptools      65.6.3
six             1.16.0
wheel           0.38.4

The current set of dependencies is most similar to the combined ui,dmf,grid,omlt targets:

pip install ".[ui,dmf,grid,omlt]"

Package                  Version
------------------------ ----------
addheader                0.3.2
anyio                    3.6.2
argon2-cffi              21.3.0
argon2-cffi-bindings     21.2.0
arrow                    1.2.3
asttokens                2.2.1
attrs                    22.2.0
backcall                 0.2.0
beautifulsoup4           4.11.2
bleach                   6.0.0
certifi                  2022.12.7
cffi                     1.15.1
charset-normalizer       3.1.0
click                    8.1.3
colorama                 0.4.6
comm                     0.1.2
contourpy                1.0.7
coramin                  0.1.1
cycler                   0.11.0
debugpy                  1.6.6
decorator                5.1.1
defusedxml               0.7.1
et-xmlfile               1.1.0
exceptiongroup           1.1.1
executing                1.2.0
fastjsonschema           2.16.3
fonttools                4.39.0
fqdn                     1.5.1
gridx-egret              0.5.3
gridx-prescient          2.2.1
idaes-pse                2.0.0.dev3
idna                     3.4
iniconfig                2.0.0
ipykernel                6.21.3
ipython                  8.11.0
ipython-genutils         0.2.0
ipywidgets               8.0.4
isoduration              20.11.0
jedi                     0.18.2
Jinja2                   3.1.2
jsonpointer              2.3
jsonschema               4.17.3
jupyter                  1.0.0
jupyter_client           8.0.3
jupyter-console          6.6.3
jupyter_core             5.2.0
jupyter-events           0.6.3
jupyter_server           2.4.0
jupyter_server_terminals 0.4.4
jupyterlab-pygments      0.2.2
jupyterlab-widgets       3.0.5
kiwisolver               1.4.4
lxml                     4.9.2
MarkupSafe               2.1.2
matplotlib               3.7.1
matplotlib-inline        0.1.6
mistune                  2.0.5
nbclassic                0.5.3
nbclient                 0.7.2
nbconvert                7.2.10
nbformat                 5.7.3
nest-asyncio             1.5.6
networkx                 3.0
notebook                 6.5.3
notebook_shim            0.2.2
numpy                    1.24.2
omlt                     1.1
openpyxl                 3.1.2
packaging                23.0
pandas                   1.5.3
pandocfilters            1.5.0
parso                    0.8.3
pexpect                  4.8.0
pickleshare              0.7.5
Pillow                   9.4.0
Pint                     0.20.1
pip                      23.0.1
platformdirs             3.1.1
pluggy                   1.0.0
ply                      3.11
prometheus-client        0.16.0
prompt-toolkit           3.0.38
psutil                   5.9.4
ptyprocess               0.7.0
pure-eval                0.2.2
pycparser                2.21
Pygments                 2.14.0
Pyomo                    6.5.0
pyparsing                3.0.9
PyPDF2                   3.0.1
pyrsistent               0.19.3
pytest                   7.2.2
python-dateutil          2.8.2
python-json-logger       2.0.7
pytz                     2022.7.1
PyYAML                   6.0
pyzmq                    25.0.1
qtconsole                5.4.1
QtPy                     2.3.0
requests                 2.28.2
rfc3339-validator        0.1.4
rfc3986-validator        0.1.1
scipy                    1.10.1
seaborn                  0.12.2
Send2Trash               1.8.0
setuptools               65.6.3
six                      1.16.0
sniffio                  1.3.0
soupsieve                2.4
stack-data               0.6.2
terminado                0.17.1
tinycss2                 1.2.1
tinydb                   4.7.1
tomli                    2.0.1
tornado                  6.2
traitlets                5.9.0
uri-template             1.2.0
urllib3                  1.26.15
wcwidth                  0.2.6
webcolors                1.12
webencodings             0.5.1
websocket-client         1.5.1
wheel                    0.38.4
widgetsnbextension       4.0.5
xlrd                     2.0.1

Changes proposed in this PR:

Split non-core dependencies in categories selectable as extras_require targets

Caveats

~~Most likely causes widespread test failures~~ Tests are now passing
~~Would still need to update the documentation~~ Documentation has been updated (I might have missed something)

Decisions

After some deliberation, it was decided not to exposes a catch-all extras_require target (e.g. complete, all). This follows the "explicit is better than implicit" principle and requires downstream clients to pro-actively "opt in" to the sets of optional dependencies they need

For reviewers

Check documentation, especially installation steps
- I've done a first pass based on what I could find, but it's entirely possible that I've missed something

Legal Acknowledgement

By contributing to this software project, I agree to the following terms and conditions for my contribution:

I agree my contributions are submitted under the license terms described in the LICENSE.txt file at the top level of this directory.
I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

setup.py

ksbeattie · 2023-03-23T18:33:33Z

This is waiting for #1138

This reverts commit 1176954.

"Explicit is better than implicit"

andrewlee94

A couple of comments that I wanted to check on, but otherwise this is looking good to me.

.github/workflows/integration.yml

andrewlee94 · 2023-05-23T12:13:26Z

docs/how_to_guides/versioned_idaes_install.rst

-   pip install idaes-pse==1.13
+   pip install idaes-pse==2.0

 * To get the latest version from the GitHub main branch ::


One thing I do not see here is instructions on how to get the optional dependencies; this is one of our user facing sets of install instructions so we probably need to cover it here (or point to the README for more details and to avoid the possibility of conflicting instrucitons).

Good point. I agree that the optional dependencies instructions should be included in the user-facing installation material. I think I'll copy what's in the README to the Sphinx docs, and we can revisit later if the repetition becomes a problem.

I've added a How-to page on optional dependencies: https://idaes-pse--1133.org.readthedocs.build/en/1133/how_to_guides/opt_dependencies.html

andrewlee94

Looks good to me.

ksbeattie

LGTM

* Remove extra indentation * Use hyphen in package distribution name * Update minimum Python version * Reorganize optional dependencies * Introduce requirements-dev.txt file * Try removing idaes-pse from requirement file * Try using IDAES/idaes-pse#1133 for dev requirements * Add dependencies for surrogates target * Add (temporarily) dmf target for convergence_base * Rename target to omlt * Remove dmf extras_require * Try removing setup.py * Remove ref to merged PR * Add test for idaes_examples.browse.find_notebook_dir() * Add gui target to dev dependencies * Add requirement for importlib_resources backport for 3.8

lbianchi-lbl added 6 commits March 13, 2023 18:44

Remove dependencies without any Ctrl-F-able imports

04ccdb1

Try removing potentially unneeded version constraints

4440b8e

Use list instead of unsupported set for columns in DataFrame constructor

d209e51

Move sympy to optional dependencies

15954e0

Remove backport package that has been obsolete since Python 3.3

382e6d9

Split optional dependencies in categories

e9672a0

lbianchi-lbl self-assigned this Mar 15, 2023

lbianchi-lbl added 2 commits March 15, 2023 19:48

Format with Black

b360707

Add pint to required dependencies

af9aea8

andrewlee94 reviewed Mar 16, 2023

View reviewed changes

setup.py Show resolved Hide resolved

Try Pyomo implementation of topical optional dependencies

4c4f943

ksbeattie added the Priority:High High Priority Issue or PR label Mar 16, 2023

lbianchi-lbl and others added 17 commits April 5, 2023 19:37

Split optional dependencies in categories

043644e

Format with Black

b7ab623

Add pint to required dependencies

81fae89

Try Pyomo implementation of topical optional dependencies

17fa84e

Add pytest.importorskip() directives to DMF test modules

1176954

Move colorama import to function scope

bee7f4a

Track updates to Prescient requirement

6d4026d

Try handling optional imports at the conftest.py level

311e926

Revert "Add pytest.importorskip() directives to DMF test modules"

866cf75

This reverts commit 1176954.

Merge remote-tracking branch 'lbianchi-lbl/issue-705-2' into issue-705-2

7157827

Fix accidental tuplification

55e360a

Remove too generic 'optional' target

db29fb0

Use appropriate version of Pyomo

fe160b4

Add version constraint for ipython from IDAES#1179

1aad67e

Remove unneeded global variable

c342821

Merge branch 'main' into issue-705-2

5682ee3

Update dev requirements to track changes in extras_require targets

f42638f

lbianchi-lbl added 3 commits May 19, 2023 18:41

Remove catch-all extras_require targets

caff30d

"Explicit is better than implicit"

Merge branch 'main' into issue-705-2

1db7264

Update Sphinx docs for pip installation

bebd429

lbianchi-lbl marked this pull request as ready for review May 22, 2023 15:04

lbianchi-lbl requested review from dangunter, eslickj and ksbeattie as code owners May 22, 2023 15:04

lbianchi-lbl requested a review from andrewlee94 May 22, 2023 21:07

andrewlee94 reviewed May 23, 2023

View reviewed changes

Andrew Lee and others added 5 commits May 23, 2023 08:14

Merge branch 'main' into issue-705-2

b4075db

Replace outdated extras_require targets in CI workflow

d9e6ccd

Merge remote-tracking branch 'lbianchi-lbl/issue-705-2' into issue-705-2

c6463e2

Add how-to guide for installing optional dependencies

921ff1e

Add ref anchors

f37a2a6

lbianchi-lbl requested a review from andrewlee94 May 23, 2023 17:35

lbianchi-lbl added the CI:run-integration triggers_workflow: Integration label May 23, 2023

idaes-build removed the CI:run-integration triggers_workflow: Integration label May 23, 2023

andrewlee94 approved these changes May 23, 2023

View reviewed changes

lbianchi-lbl added the CI:run-integration triggers_workflow: Integration label May 23, 2023

idaes-build removed the CI:run-integration triggers_workflow: Integration label May 23, 2023

ksbeattie approved these changes May 25, 2023

View reviewed changes

Merge branch 'main' into issue-705-2

a70c11e

lbianchi-lbl merged commit 0651175 into IDAES:main May 26, 2023

lbianchi-lbl added a commit to lbianchi-lbl/examples that referenced this pull request May 26, 2023

Try using IDAES/idaes-pse#1133 for dev requirements

c74d8b1

lbianchi-lbl added a commit to lbianchi-lbl/examples that referenced this pull request Jun 2, 2023

Try using IDAES/idaes-pse#1133 for dev requirements

c6aea97

This was referenced Jun 8, 2023

Unable to run pytest on a single test file #1200

Closed

[WIP] Price-taker framework for DISPATCHES #1201

Closed

This was referenced Jun 21, 2023

Update idaes-pse requirement in preparation to 2.1 release watertap-org/watertap#1034

Merged

Move UI visualizer code out of this repo #1146

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Spring cleaning for dependencies, Pt. 2: The Tree-Chopping#1133

Spring cleaning for dependencies, Pt. 2: The Tree-Chopping#1133
lbianchi-lbl merged 56 commits intoIDAES:mainfrom
lbianchi-lbl:issue-705-2

lbianchi-lbl commented Mar 15, 2023 •

edited

Loading

Uh oh!

Uh oh!

ksbeattie commented Mar 23, 2023

Uh oh!

andrewlee94 left a comment

Uh oh!

Uh oh!

andrewlee94 May 23, 2023

Uh oh!

lbianchi-lbl May 23, 2023

Uh oh!

lbianchi-lbl May 23, 2023

Uh oh!

andrewlee94 left a comment

Uh oh!

ksbeattie left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

lbianchi-lbl commented Mar 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Resolves: #705

Motivation

Changes proposed in this PR:

Caveats

Decisions

For reviewers

Legal Acknowledgement

Uh oh!

Uh oh!

ksbeattie commented Mar 23, 2023

Uh oh!

andrewlee94 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

andrewlee94 May 23, 2023

Choose a reason for hiding this comment

Uh oh!

lbianchi-lbl May 23, 2023

Choose a reason for hiding this comment

Uh oh!

lbianchi-lbl May 23, 2023

Choose a reason for hiding this comment

Uh oh!

andrewlee94 left a comment

Choose a reason for hiding this comment

Uh oh!

ksbeattie left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

lbianchi-lbl commented Mar 15, 2023 •

edited

Loading