Skip to content

Add GAMS to MacOS#1281

Merged
jsiirola merged 20 commits intoPyomo:masterfrom
mrmundt:mac_gams
Feb 12, 2020
Merged

Add GAMS to MacOS#1281
jsiirola merged 20 commits intoPyomo:masterfrom
mrmundt:mac_gams

Conversation

@mrmundt
Copy link
Copy Markdown
Contributor

@mrmundt mrmundt commented Jan 30, 2020

Fixes #1279

Summary/Motivation:

GAMS is represented in Windows and Linux testing. We need to add it to MacOS.

Changes proposed in this PR:

  • Add GAMS installation to MacOS
  • GAMS version 29.1.0 (older distributions do not install on Mac 10.15)
  • Fix Windows/Linux path variables for GAMS solver

Legal Acknowledgement

By contributing to this software project, I agree to the following terms and conditions for my contribution:

  1. I agree my contributions are submitted under the BSD license.
  2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

@mrmundt mrmundt requested a review from jsiirola January 30, 2020 21:18
@codecov
Copy link
Copy Markdown

codecov bot commented Jan 31, 2020

Codecov Report

Merging #1281 into master will decrease coverage by 2.45%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master   #1281      +/-   ##
=========================================
- Coverage   70.95%   68.5%   -2.46%     
=========================================
  Files         534     474      -60     
  Lines       81004   75733    -5271     
=========================================
- Hits        57479   51880    -5599     
- Misses      23525   23853     +328
Impacted Files Coverage Δ
pyomo/core/base/param.py 82.74% <100%> (+1.12%) ⬆️
pyomo/pysp/plugins/csvsolutionwriter.py 24.24% <0%> (-72.73%) ⬇️
pyomo/opt/parallel/pyro.py 18.34% <0%> (-71.56%) ⬇️
pyomo/pysp/solvers/benders.py 11.34% <0%> (-70.45%) ⬇️
pyomo/solvers/plugins/solvers/direct_solver.py 12.22% <0%> (-70%) ⬇️
pyomo/pysp/lagrangeMorePR.py 5.54% <0%> (-68.98%) ⬇️
pyomo/contrib/pynumero/extensions/asl.py 20.34% <0%> (-65.7%) ⬇️
pyomo/pysp/scenariotree/action_manager_pyro.py 18.12% <0%> (-64.92%) ⬇️
pyomo/solvers/plugins/solvers/CPLEX.py 20.7% <0%> (-64.19%) ⬇️
pyomo/solvers/plugins/smanager/pyro.py 22.22% <0%> (-61.12%) ⬇️
... and 141 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 87f1182...e557faf. Read the comment docs.

@mrmundt mrmundt changed the title Add GAMS to MacOS [WIP] Add GAMS to MacOS Jan 31, 2020
@mrmundt
Copy link
Copy Markdown
Contributor Author

mrmundt commented Jan 31, 2020

@jsiirola - Fun fact: I checked and the same issue was happening with Ubuntu and Windows. I have fixed the issues.
In addition, I tried a one-older GAMS distribution for Mac (29.1.0). It installs and runs without a license file, so I've changed the workflow to incorporate this while we wait on #1280.

@mrmundt mrmundt requested a review from jsiirola January 31, 2020 20:58
@jsiirola
Copy link
Copy Markdown
Member

We are getting closer. We are still seeing:

test_file_removal_py (pyomo.solvers.tests.checks.test_GAMS.GAMSTests) ... SKIP: The 'gams' python bindings are not available

It looks like we can install them with:

cd gams/gams29.1_linux_x64_64_sfx/apifiles/Python/api && python setup.py install

(substituting api_34, pai_36, and api_37 for those versions of Python... 3.5 and 3.8 might be able to get away with (automatically) rebuilding the bindings drom the api directory?)

@mrmundt
Copy link
Copy Markdown
Contributor Author

mrmundt commented Jan 31, 2020

We are getting closer. We are still seeing:

test_file_removal_py (pyomo.solvers.tests.checks.test_GAMS.GAMSTests) ... SKIP: The 'gams' python bindings are not available

It looks like we can install them with:

cd gams/gams29.1_linux_x64_64_sfx/apifiles/Python/api && python setup.py install

(substituting api_34, pai_36, and api_37 for those versions of Python... 3.5 and 3.8 might be able to get away with (automatically) rebuilding the bindings drom the api directory?)

Another one that is happening for all OS'! Interesting point there - you can just give it the flag -noCheck to power through without worrying about the Python version. The original api folder looks like it's intended for Py27. I tried python setup.py install -noCheck on my local machine (which runs Py37) and it built, so it might work out for Py35/38.

I think, in terms of "closeness", we might want to try the api_34 bindings for Py35 and api_37 for Py38. Thoughts?

@jsiirola
Copy link
Copy Markdown
Member

I think, in terms of "closeness", we might want to try the api_34 bindings for Py35 and api_37 for Py38. Thoughts?

No - the code for the various versions should be the same. The difference is how the compiled module libraries are linked. So, api_34 should only work for Python 3.4.x. When we use api's setup.py for other python versions, it should be recognizing that the Python version is different and regenerating / recompiling the libraries from scratch (the point of api_3? is that on those platforms, no compilation is necessary).

@mrmundt
Copy link
Copy Markdown
Contributor Author

mrmundt commented Jan 31, 2020

I think, in terms of "closeness", we might want to try the api_34 bindings for Py35 and api_37 for Py38. Thoughts?

No - the code for the various versions should be the same. The difference is how the compiled module libraries are linked. So, api_34 should only work for Python 3.4.x. When we use api's setup.py for other python versions, it should be recognizing that the Python version is different and regenerating / recompiling the libraries from scratch (the point of api_3? is that on those platforms, no compilation is necessary).

Gotcha! Makes sense to me. I'll work on that.

@mrmundt
Copy link
Copy Markdown
Contributor Author

mrmundt commented Feb 3, 2020

We are getting closer. We are still seeing:

test_file_removal_py (pyomo.solvers.tests.checks.test_GAMS.GAMSTests) ... SKIP: The 'gams' python bindings are not available

It looks like we can install them with:

cd gams/gams29.1_linux_x64_64_sfx/apifiles/Python/api && python setup.py install

(substituting api_34, pai_36, and api_37 for those versions of Python... 3.5 and 3.8 might be able to get away with (automatically) rebuilding the bindings drom the api directory?)

@jsiirola - It turns out, this issue exists in ALL of our testing suites. If you look at Appveyor, Jenkins, and Travis, the GAMS python binding tests are skipped in every single one. I am experimenting with getting it working on Ubuntu first (I was able to get them working on my local RHEL).

@mrmundt mrmundt changed the title [WIP] Add GAMS to MacOS Add GAMS to MacOS Feb 12, 2020
@mrmundt
Copy link
Copy Markdown
Contributor Author

mrmundt commented Feb 12, 2020

@jsiirola @allevin @blnicho - This is finally complete. The Python bindings don't work for Python 3.5 and 3.8 on Windows, but that's it. Everything else is fine.

Related to this: It turns out there is (at least on Windows) a bug with conda that occurs when a package fails to install. I submitted an issue to conda for it.

Copy link
Copy Markdown
Member

@jsiirola jsiirola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is OK, but we might consider changing the Python version checks for Linux/OSX.

@jsiirola jsiirola merged commit f8a3a1e into Pyomo:master Feb 12, 2020
@mrmundt mrmundt deleted the mac_gams branch February 27, 2020 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add GAMS to OSX Github Actions

4 participants