11/17/2019
Ali M. Sahlodin
Department of Chemical Engineering
AmirKabir University of Technology
1397 S.H
Parametric Dynamic Systems
Parametric sensitivities
Finite differences
Forward sensitivities
Copyright © Ali M. Sahlodin, Dept. of Chemical Engineering, AmirKabir Univ. of Tech. 2
1
11/17/2019
Optimization algorithms need
sensitivities of objective/constraints
w.r.t parameters
Copyright © Ali M. Sahlodin, Dept. of Chemical Engineering, AmirKabir Univ. of Tech. 3
Dynamic optimization (DO) problem
ABC
Maximize final production of B
Find optimal temperature
very simplified DO! Only one parameter.
Copyright © Ali M. Sahlodin, Dept. of Chemical Engineering, AmirKabir Univ. of Tech. 4
2
11/17/2019
Form the system of forward sensitivities
t ; p) f (t , x(t; p), p)
x(
x p j (t ; p) f x (t , x(t; p), p)x p j (t; p) f p j (t , x(t; p), p)
x p j (t0 ; p) h p j (p)
The system parameter is temperature θ
Copyright © Ali M. Sahlodin, Dept. of Chemical Engineering, AmirKabir Univ. of Tech. 5
Forward sensitivity equations
dk
(C A ) 1 C A2 2k1C A (C A )
d
dk dk
(C B ) 1 C A2 2k1C A (C A ) 2 CB k2 (CB )
d d
dki b b
ai ( 2 ) exp( )
d R R
at t 0 (C A ) (CB ) 0
1. Integrate the above sensitivity system together
with the original ODEs
2. Evaluate sensitivity for the objective function
Copyright © Ali M. Sahlodin, Dept. of Chemical Engineering, AmirKabir Univ. of Tech. 6
3
11/17/2019
•Finite differences: (np+1) integrations of nx eqs. Computationally expensive
•Forward sensitivity: 1 integration of (np+1)nx eqs. for even small plant models
What sensitivities do we actually need?
Sensitivities of these
are finally needed.
State sensitivities are
intermediate results and
not required per se.
Can we bypass state sensitivities and directly calculate sensitivities of
objective/constraints?
Copyright © Ali M. Sahlodin, Dept. of Chemical Engineering, AmirKabir Univ. of Tech. 7
Objective function J (p) : (x(tf ; p), p)
dJ dx
Sensitivity at p j : (p) (x(tf ; p), p) (x(tf ; p), p) (tf ; p)
dp j p j x dp j
1. Adjoin the differential equations to the objective function
tf
J (p) : (x(tf ; p), p) λ(t )T f (t , x(t; p), p) x(
t; p) dt
t0
2. Differentiate the augmented functional
dJ dJ
(p) (p) p j (x(tf ), p) x (x(tf ), p)T x p j (tf )
dp j dp j
t
λ(t )T f p j (t , x(t ), p) f x (t , x(t ), p)x p j (t ) x p j (t ) dt
f
t0
Copyright © Ali M. Sahlodin, Dept. of Chemical Engineering, AmirKabir Univ. of Tech. 8
4
11/17/2019
3. Apply integration by part
tf t tf
t λ(t )T x p j (t )dt λ(t )T x p j (t ) λ( t )T x (t )dt
f
0
t0 t0 pj
4. Substitute
dJ dJ
(p) (p) p j (x(tf ), p) λ(t0 )T h p j (p)
dp j dp j
T
x (x(tf ), p)-λ(t f ) x p j (tf )
t T
t ) x ) dt
(λ(t )T f p j (t , x(t ), p) f x (t , x(t ), p)T λ(t ) λ(
f
t0 pj
Copyright © Ali M. Sahlodin, Dept. of Chemical Engineering, AmirKabir Univ. of Tech. 9
5. To eliminate state sensitivities, their multipliers must be zero.
λ * (t ) f x (t , x(t ), p)T λ* (t ) λ* (t f ) x (x(tf ), p)
5. Now, sensitivities of objective function is simplified as
Direct influence of
parameter on objective
dJ dJ
(p) (p) p j (x(tf ), p) λ* (t0 )T h p j (p)
dp j dp j
tf
(λ* (t )T f p j (t , x(t ), p)) dt Influence of initial conditions
t0
Influence of system
dynamics
Copyright © Ali M. Sahlodin, Dept. of Chemical Engineering, AmirKabir Univ. of Tech. 10
5
11/17/2019
1. Integrate system dynamics (forward in time)
t ; p) f (t , x(t; p), p) x(t0 ) h(p)
x(
Store state values at time points
t0 t1 t f
2. Integrate the adjoint system (backward in time)
λ * (t ) f x (t , x(t ), p)T λ* (t ) λ* (t f ) x (x(tf ), p)
q j =-λ(t )T f p j (t , x(t ), p) q j (t f )=0, j 1,, n p
3. Compute gradients
dJ
(p) p j (x(tf ), p) λ* (t0 )T h p j (p) q j (t0 )
dp j
Copyright © Ali M. Sahlodin, Dept. of Chemical Engineering, AmirKabir Univ. of Tech. 11
Maximize J ( ) CB2 a 2
Construct adjoint equations
λ * (t ) f x (t , x(t ), p)T λ* (t ) λ* (t f ) x (x(tf ), p)
q j =-λ(t )T f p j (t , x(t ), p) q j (t f )=0, j 1,, n p
Copyright © Ali M. Sahlodin, Dept. of Chemical Engineering, AmirKabir Univ. of Tech. 12
6
11/17/2019
Adjoint equations
1* (t ) T
2k1C A 0 1 (t )
* 1* (t f ) 0
,
2* (t ) 2k1C A k2 2* (t ) 2C (
2 (t f ) B f
*
t )
T
dk1 2
d C A 1* (t )
q =- , q(t f )=0
dk1 C 2 dk2 C 2* (t )
d A d B
dJ
( ) 2a λ* (t0 )T h p j (p) q(t0 )
d
0
Copyright © Ali M. Sahlodin, Dept. of Chemical Engineering, AmirKabir Univ. of Tech. 13
Adjoint Sensitivity
More efficient than forward sensitivity when there is a
large number of parameters and few functionals.
Requires more memory for storing state values in the
forward integration.
With variable step integrators, interpolation is needed to
evaluate state values for the adjoint system.
Does NOT provide state sensitivities.
Does NOT provide sensitivity at intermediate times.
Copyright © Ali M. Sahlodin, Dept. of Chemical Engineering, AmirKabir Univ. of Tech. 14
7
11/17/2019
Partial derivatives should be obtained via
symbolic or automatic differentiation
Avoid finite differences if you can.
Generally, sensitivity results are local: only
valid at p .
Copyright © Ali M. Sahlodin, Dept. of Chemical Engineering, AmirKabir Univ. of Tech. 15
Solver Website License Forward Adjoint
CVODES [Link] Free Yes Yes
s/
IDAS [Link] Free Yes Yes
s/
DASPK3.1 [Link] Free for acad. Yes
.edu/
~cse/[Link]
DASPKadjoint [Link] Free for acad. Yes
.edu/
~cse/[Link]
Copyright © Ali M. Sahlodin, Dept. of Chemical Engineering, AmirKabir Univ. of Tech. 16
