Skip to content

Numpy expression integration (part 1)#2027

Merged
michaelbynum merged 12 commits intoPyomo:mainfrom
jsiirola:numpy-integration-part1
Jun 28, 2021
Merged

Numpy expression integration (part 1)#2027
michaelbynum merged 12 commits intoPyomo:mainfrom
jsiirola:numpy-integration-part1

Conversation

@jsiirola
Copy link
Copy Markdown
Member

Fixes #685, fixes #31

Summary/Motivation:

This is the initial step toward integrating numpy with the Pyomo expression system. This provides basic support for generating Pyomo expressions contained in NumPy ndarrays.

It is by no means complete (there will be at least 3 more parts to finish the integration), but I felt breaking this up into chunks will make it easier to review. However, this PR will support the following example:

>>> import numpy as np
>>> from pyomo.environ import *
>>> m = ConcreteModel()
>>> m.x = Var([0,1,2,3])
>>> A = np.array([[1, 2, 3, 4], [5, 6, 7, 8]])
>>> b = np.array([10, 20])
>>> m.c = Constraint([0,1], expr=A @ m.x <= b)
>>> m.pprint()
2 Set Declarations
    c_index : Size=1, Index=None, Ordered=Insertion
        Key  : Dimen : Domain : Size : Members
        None :     1 :    Any :    2 : {0, 1}
    x_index : Size=1, Index=None, Ordered=Insertion
        Key  : Dimen : Domain : Size : Members
        None :     1 :    Any :    4 : {0, 1, 2, 3}

1 Var Declarations
    x : Size=4, Index=x_index
        Key : Lower : Value : Upper : Fixed : Stale : Domain
          0 :  None :  None :  None : False :  True :  Reals
          1 :  None :  None :  None : False :  True :  Reals
          2 :  None :  None :  None : False :  True :  Reals
          3 :  None :  None :  None : False :  True :  Reals

1 Constraint Declarations
    c : Size=2, Index=c_index, Active=True
        Key : Lower : Body                              : Upper : Active
          0 :  -Inf :   x[0] + 2*x[1] + 3*x[2] + 4*x[3] :  10.0 :   True
          1 :  -Inf : 5*x[0] + 6*x[1] + 7*x[2] + 8*x[3] :  20.0 :   True

4 Declarations: x_index x c_index c

Please note that this implementation is NOT intended to be "numpy-fast": we still generate the same expanded Pyomo expression trees that we always have. However, this is intended to lay the groundwork for a fast "matrix-based" expression system later.

Changes proposed in this PR:

  • add support for casting some IndexedComponents into ndarray objects
  • add recognition for ndarray in Initializer()

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

  1. I agree my contributions are submitted under the BSD license.
  2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

@codecov
Copy link
Copy Markdown

codecov bot commented Jun 24, 2021

Codecov Report

Merging #2027 (4d77821) into main (6216ead) will increase coverage by 0.03%.
The diff coverage is 93.33%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2027      +/-   ##
==========================================
+ Coverage   82.71%   82.74%   +0.03%     
==========================================
  Files         592      592              
  Lines       73355    73522     +167     
==========================================
+ Hits        60675    60837     +162     
- Misses      12680    12685       +5     
Impacted Files Coverage Δ
pyomo/core/base/indexed_component.py 88.03% <91.30%> (+0.17%) ⬆️
pyomo/core/expr/numvalue.py 94.37% <91.66%> (-0.10%) ⬇️
pyomo/core/base/param.py 80.88% <100.00%> (ø)
pyomo/core/base/util.py 99.47% <100.00%> (+0.01%) ⬆️
pyomo/core/base/var.py 88.96% <100.00%> (ø)
pyomo/contrib/gdpopt/iterate.py 85.00% <0.00%> (-6.67%) ⬇️
pyomo/contrib/gdpopt/branch_and_bound.py 72.47% <0.00%> (-3.22%) ⬇️
pyomo/contrib/gdpbb/GDPbb.py 23.31% <0.00%> (+0.12%) ⬆️
pyomo/contrib/fbbt/fbbt.py 95.08% <0.00%> (+0.37%) ⬆️
pyomo/core/base/constraint.py 89.97% <0.00%> (+0.47%) ⬆️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6216ead...4d77821. Read the comment docs.

Copy link
Copy Markdown
Contributor

@michaelbynum michaelbynum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, this looks great. I had one suggestion for the tests.

m.x = Var()
e = np.float64(5) * m.x
self.assertIs(type(e), MonomialTermExpression)
self.assertEqual(str(e), "5.0*x")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would recommend using the compare_expressions function from pyomo/core/expr/compare.py throughout these tests.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. Done.

@michaelbynum michaelbynum merged commit ffecc01 into Pyomo:main Jun 28, 2021
@jsiirola jsiirola deleted the numpy-integration-part1 branch July 2, 2021 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot do simple scalar variable multiplication... Bug using numpy float values in Pyomo expressions.

2 participants