Skip to content

Direct and persistent interfaces to Xpress#1443

Merged
blnicho merged 22 commits intoPyomo:masterfrom
bknueven:xpress_solver
Jul 20, 2020
Merged

Direct and persistent interfaces to Xpress#1443
blnicho merged 22 commits intoPyomo:masterfrom
bknueven:xpress_solver

Conversation

@bknueven
Copy link
Copy Markdown
Contributor

@bknueven bknueven commented May 15, 2020

Fixes N/A

Summary/Motivation:

Xpress is a commonly used commercial MIP solver. This change adds direct and persistent interfaces to Xpress, leveraging Xpress's python interface. Both are largely based on their Gurobi counterparts. Note that FICO distributes Xpress with a community license (5000 variable/constraint limit) on PyPI.

Changes proposed in this PR:

  • New solvers 'xpress_direct' and 'xpress_persistent'
  • Tests for xpress_persistent
  • DONE: Modify Pyomo's test harness to run Xpress

Limitations:

  • Xpress has methods for solving general NLPs and MINLPs. These new interfaces are limited to problems with linear and quadratic constraints/objectives.

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

  1. I agree my contributions are submitted under the BSD license.
  2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

@codecov
Copy link
Copy Markdown

codecov bot commented May 15, 2020

Codecov Report

Merging #1443 into master will increase coverage by 0.55%.
The diff coverage is 73.51%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1443      +/-   ##
==========================================
+ Coverage   72.39%   72.94%   +0.55%     
==========================================
  Files         612      619       +7     
  Lines       86031    90450    +4419     
==========================================
+ Hits        62279    65978    +3699     
- Misses      23752    24472     +720     
Impacted Files Coverage Δ
pyomo/solvers/plugins/solvers/XPRESS.py 27.75% <33.33%> (+0.26%) ⬆️
pyomo/solvers/plugins/solvers/xpress_direct.py 72.81% <72.81%> (ø)
pyomo/solvers/plugins/solvers/xpress_persistent.py 87.27% <87.27%> (ø)
pyomo/contrib/benders/benders_cuts.py 88.53% <100.00%> (+0.10%) ⬆️
pyomo/solvers/plugins/solvers/__init__.py 100.00% <100.00%> (ø)
...trib/pynumero/algorithms/solvers/cyipopt_solver.py 59.55% <0.00%> (-29.78%) ⬇️
...b/pynumero/algorithms/solvers/pyomo_ext_cyipopt.py 62.16% <0.00%> (-28.63%) ⬇️
pyomo/contrib/sensitivity_toolbox/sens.py 70.94% <0.00%> (-26.71%) ⬇️
pyomo/solvers/plugins/solvers/gurobi_persistent.py 55.43% <0.00%> (-12.97%) ⬇️
pyomo/pysp/scenariotree/action_manager_pyro.py 80.95% <0.00%> (-1.79%) ⬇️
... and 36 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 84e8a4e...2c72213. Read the comment docs.

@blnicho
Copy link
Copy Markdown
Member

blnicho commented May 18, 2020

I opened #1444 to track adding Xpress to our testing infrastructure

@blnicho
Copy link
Copy Markdown
Member

blnicho commented May 19, 2020

@bknueven Xpress has been added to our GitHub Actions and Travis tests

@bknueven
Copy link
Copy Markdown
Contributor Author

It looks like xpress 8.9 is causing the python 3.6 and 3.7 tests to segfault.

@bknueven
Copy link
Copy Markdown
Contributor Author

bknueven commented Jun 5, 2020

I spent some time attempting to correct the segfault issue on Python 3.6/3.7. I'm able to reproduce it on my system on Python 3.7. I was ultimately unsuccessful -- though I narrowed it down to a test in pyomo-model-libraries and the xpress_persistent test, the behavior was non-deterministic on my system.

I also attempted to re-factor the Xpress direct/persistent interfaces to use as few Xpress python objects as possible, to no avail.

Given that the issue only comes up on exit, and the tests still pass, would an acceptable solution be to remove the xpress installation from python 3.6/3.7? I was hoping the new release of Xpress (8.9.2) would resolve this issue, but that's clearly not the case.

Copy link
Copy Markdown
Contributor

@qtothec qtothec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple typos and a broader question

Comment on lines +86 to +89
# TODO: this isn't a limit of XPRESS, which implements an SLP
# method for NLPs. But it is a limit of *this* interface
self._max_obj_degree = 2
self._max_constraint_degree = 2
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you suggest creation of a new interface, or modification of this one to support SLP?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this one could be modified to support SLP. Xpress has support in its Python expression system for trigonometric functions, exponents and logarithms, and general polynomials.

In addition modifying the method which get information from generate_standard_repn, probably some of the solution fetching logic would need to be revisited as well.

That said, this implementation is largely based on its Gurobi and CPLEX counterparts, which don't have such functionality. I would definitely need help to make it work.

@jsiirola
Copy link
Copy Markdown
Member

jsiirola commented Jun 5, 2020

@bknueven: which test is the most likely to cause a segfault? I can try and take a look at it and see if I can reproduce it here...

@bknueven
Copy link
Copy Markdown
Contributor Author

bknueven commented Jun 5, 2020

@jsiirola here's the subset of tests that reliably reproduce it on my system:

nosetests -v --with-xunit --xunit-file=TEST-pyomo.xml "--eval-attr=nightly and (not fragile)" ../pyomo-model-libraries/cute/testCUTE_baseline.py pyomo/solvers/tests/checks/test_xpress_persistent.py 

Separately they do fine. I'm running macOS with python 3.7.7.

@jsiirola
Copy link
Copy Markdown
Member

jsiirola commented Jun 5, 2020

Hmmm. So far I have not been able to reproduce locally with Python 3.7.5 on RHEL6 / RHEL7 and xpress 8.9.0. I will try building Python 3.7.7 and see if that makes a difference.

Copy link
Copy Markdown
Contributor

@DLWoodruff DLWoodruff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code has previously been reviewed and now the tests are passing, so I recommend merging.

@blnicho blnicho merged commit 0725366 into Pyomo:master Jul 20, 2020
@bknueven bknueven deleted the xpress_solver branch July 20, 2020 19:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants