Practical Adaptive Control:

Foundations, Verification, and Validation

Anuradha Annaswamy
Active-Adaptive Control Laboratory
Department of Mechanical Engineering
Outline
• Foundations
– Parameter Adaptation
– Robust parameter estimation
– Nonlinear Dynamics
– Large Delays
– Unmatched uncertainties

• Verification: Theoretical tools

– Integration with a Baseline
– Anti-windup solutions
– Optimization of fixed components
– Embedded co-designs of Implementations
– Delay margins

• Validation
– High Fidelity Simulations
– Hardware-in-the-loop (HIL) Tests
– Stand-alone Hardware Tests
Foundations: Adaptive Control
Adaptive Control

Model Structure Control input

Adaptive
Controller
On-line information
Parameter estimate

The design of a self-tuning controller to

cope with parametric uncertainties.
Adaptive Systems:
A bit of history
1960s: A Brave New Era
• Several symposia:
P.C. Gregory et al., Self-adaptive flight control symposium, WPAFB, 1959
P.H. Hammond, Theory of Self-Adaptive Control Systems, IFAC symposium, 1970

• “Three-legged milking stool”:

Propulsion, Aerodynamics, GNC, with
the latter emphasizing adaptive control systems
• Earliest books on Adaptive Control: 1959-62
Bellman, Widrow, Mishkin and Braun,..
• Whitaker’s MIT Rule: Gradient-descent
• GE and Honeywell’s AFCS (1959)
1970s: Stability Framework
• Cautionary inputs:
Butchart, R.L. and B. Shackcloth, “Synthesis of MRAS by Lyapunov’s Second
Method, 1965 IFAC Symposium on Adaptive Control
Parks, P.C., “Lyapunov redesign of MRAC Systems,” IEEE Trans. Autom. Contr.,
1966.
• Lyapunov Stability:
Narendra, K.S. and P. Kudva, “…”, IEEE Systems, Man, and Cybernetics, 1972.
• Hyperstability:
I. Landau, “Adaptive Control”, Marcel Dekker, 1973.
• Dual Control:
A.A. Feldbaum, “Optimal Control Theory,” Academic Press, 1965.
• Self-Tuning Regulators:
K.J. Astrom and B. Wittenmark, “Self-Tuning Regulators,” 1976.
Problem Statement

θ : parameter, unknown
x,y,u: state, output, input

e: tracking error
θc: control parameter estimate

Goal: Find u, C1, C2 so that regulation

and tracking occur
Adaptive Control: Milestones
x = f ( x,θ , u )
y = g ( x, θ , u )

1. f,g: linear. Stability established in 1980.

2. Robustness to disturbances and unmodeled
dynamics in the ’90s.
3. f,g: nonlinear. Stability and robustness
established in 1990-2000.
 A Simple Example
• Plant: , x = Ap x + B p u Ap unknown

• A stable solution:

• An optimal solution:
•
• Adaptive control solution: u = (θ (t ) + K )x + r θ

• Plant + adaptive controller:

• Determine the rule for adjusting θ using Lyapunov methods

Adaptive Controller
• Plant + adaptive controller Am x p + B pθT (t ) x p + B p r
x p =
θ = −Γx p eT PB p

• Reference Model
Command

• Error Model =e Ame + B pθT (t ) x p

xp e
( sI − Am )
−1
θT Bp

(
e Pe + Trace θT Γ −1θ
V= T
) ⇒ V =
−eT Qe ≤ 0

 Global Stability Barbalat’s Lemma 

 Effect of Disturbances
• Plant: x p =a p x p + bp (λu + d ) a p , bp known, λ unknown
• Adaptive controller: = u θ x (t ) x p + θ r (t )r

• 
Plant + controller: p m p m p
x = a x + b r + b λ ( (θ x (t ) − θ *
x ) x p + (θ r (t ) − θ r ) r ) + bp d
*

• Reference model: = xm am xm + bm r

• Error model: e = am e + bp λ (θx (t ) x p + θr (t )r ) + bp d

Lyapunov function candidate

1 2
(
V = e + bp λ θx2 + θr2
2
( )) −e0 e0

=V 2am e 2 + 2bp ed e0 =

bp d max

≤ − | e | (| am || e | −2 | bp λ | d max )
| am | V > 0?
 Robust Parameter Estimation
• Plant: x p =a p x p + bp (λu + d ) a p , bp known,λ unknown
• Adaptive controller: = u θ x (t ) x p + θ r (t )r

• Plant + controller: 
x p = am x p + b m r + b p λ ( (θ x (t ) − θ *
x ) x p + (θ r (t ) − θ *
r ) r ) + bp
• Reference model: = xm am xm + bm r
• Error model: e = am e + bp λ (θx (t ) x p + θr (t )r ) + bp d

θ = − sgn ( bp λ ) eω − σθ
Lyapunov function candidate V > 0?
(
V = e + bp λ (θx2 + θr2 )
1 2
)
= V 2am e 2 + 2bp ed − σθθ 2

(
≤ − | e | (| am || e | −2 | bp | d max ) − σ θ θ − θ max
*
)
(
≤ − | am | (| e | −a1 ) (
+ σ | θ | −a2 ))
2
+ c 2 ≤ 0 outside an ellipse
2
Nonlinearities
1. Robustness approach:
e = Ae + BΛ (θT ω + ε 0 ( x) ) ε 0 ( x) ≤ µ x + c0
m

Adaptive law:

θ = −Γe PBω − σ f (θ , e)
T

For µ ≤ µ * , the adaptive system has

semi-globally bounded solutions.
2. Structure-based approach: Stability results are available
x = f ( x,θ ) + g ( x, θ )u for specific f and g
ˆ • back-stepping
u = h1 ( x, θ );θ = h2 ( x, u )
ˆ
• nonlinear parameterization
h1 and h2 depend on f and g .
 Delays

r(t) + e − sτ Λ Plant y(t)

Parametric
Time Delay:
Controller Uncertainty:
Signal Processing,
Actuator Failure,
Computation,
Electrical fault,
Communication
Damage

• Goal: Design a controller so that states y(t) track

commands r(t) despite the uncertainty and the
delay
 Adaptive Posicast Controller*
k m − sτ ym (t )
e
0

∫ λ (σ )u(t + σ )dσ
Rm e(t )
−τ Σ

u (t ) Zp − sτ
y p (t )
r (t ) θ 4 (t ) Σ kp e
+ Rp
+

Λ, l
Λ, l
θ(t ) = − Γe(t )ω (t − τ )
ω1
θ1T
ω2
θ2 T

Evolution of θ (ξ ) between [t − τ , t ] must be accommodated.

→ Krasovskii functional.
 t ~ 2 
[ ]≤0
0
~2
( )
V = e + θ + ∫  ∫ θ ξ dξ dν
2
V = −e(t ) α − 2τ ω (t − τ )
2
for small τ .
−τ t +ν 
* Niculescu and Annaswamy, Systems and Control Letters , vol. 49, pp. 347-358, 2003.
Large Delays: Posicast + CMRAC
• Adaptive Posicast Control (APC)***
– Explicitly accounts for known delay in the system
– Gives a time delay margin τ*
• Combined / Composite MRAC (CMRAC)*
– Uses information from 2 sources to adjust parameters: tracking error
and prediction error
– Results in smoother transients
• Bounded-Gain-Forgetting (BGF) adaptive gains**
– Adaptive gains become time-varying
– Allows faster parameter convergence, smoother parameter estimates
• Time-delay Resistant adaptive control+

* Duarte, Narendra, “A new approach to model reference adaptive control,” Int J Adaptive Control, 1988.
* Slotine, Li, “Composite adaptive control of robot manipulators,” Automatica, 1989.
** Slotine, Li, “Parameter estimation strategies for robotic applications,” ASME Winter Annual Meeting, 1987.
*** Yildiz, et al. “Adaptive posicast controller for time-delay systems with relative degree n* ≤2," Automatica, 2010.
+ Dydek, Annaswamy, Slotine, Lavretsky, “Time delay resistant adaptive control of mini-UAVs,“ IFAC TDS Workshop,
2010.
Time Delay Resistant Adaptive Control*
• Time Delay Resistant (TDR) Adaptive Control:
– Combines all of the modifications into one coherent
structure
– Has guaranteed stability via a Lyapunov-based approach

Theorem:
Given initial conditions on the adaptive parameters at t = 0 and
initial conditions on the states for –τ ≤ t ≤ 0, there exists a τ* such
that for all τ ≤ τ*, the adaptive system has bounded trajectories
for all t > 0. Time Delay Margin
£ ¡ 2 ¢ ¤
V_ · ¡ e (t) Q ¡ 2¿ ° 0 + 1 jj! (t ¡ ¿)jj PB B P e(t)
T 2 T

T
£ 2 2 2
¤
¡ eY (t) ° c ¡ ¿° c ° 0 jj! f (t ¡ ¿)jj eY (t)
Verification

Uncertainties, Modeling
Errors, Anomalies

+ Adaptive
Cmd Plant
Controller
-

Guarantee desired performance in the presence

of various unknowns
Verification Tools
Uncertainties, Modeling Errors,
Anomalies

+ Baseline
Adaptive Mag &
Cmd + Rate limits Plant

Network
Controller
Controller
-

– Integration with a Baseline

– Anti-windup solutions
– Optimization of fixed components
– Embedded co-designs of Control/Implementation
– Delay margins
Integration with Baseline Designs
Hypersonic Vehicle

• Baseline designs:
– LQR+Integral action
– Gain-scheduling (see the above example)
– Optimal controllers
• Augmentation is nontrivial, especially with partial states
accessible
Validation: X-15
 Summary of X-15 Project
• Developed a full nonlinear model of the X-15 aircraft and original
1960’s MH-96 adaptive controller
– Reproduces anomalous behavior, fatal crash from 1967

• Developed a provably correct adaptive architecture

– Comes from an understanding of history and lessons learned

• Replacement of original MH-96 controller with the provably correct

AFCS
– Results in increased robustness to unknown parameters
– 1967 crash could have been avoided

* Data from public domain sources

 Quick Comparison

MH-96 AFCS Provably Correct AFCS

 Why does it work?
• Full state feedback
– Takes into account more states
– Takes into account coupling between states
• Adaptation
– Uses adaptive law grounded in theory
• Saturation accounted for explicitly
 A Closer Look
δe δa δr
α θ11 θ12 θ13
β θ 21 θ 22 θ 23
p θ31 θ32 θ33
q θ 41 θ 42 θ 43 H, V r
r θ51 θ52 θ53 H cmd , Vcmd
δ δ nom
pi θ 61 θ 62 θ 63 H, V r δ nom = K xi x ucmd
δ ad i
qi θ 71 θ 72 θ 73 H, V r h
H cmd , Vcmd
δ δ nomH , V
δ nom = K x x
θ81 θ82 θ83
i
ucmd

24 Adaptive Parameters
ri i δ ad i
H cmd , Vcmd
δ δ nom θ = −Γ i BTpi Pe
i uω
T
δ nom = K x xh
h i
ucmd
θ
H, V

H, V
δ ad i
Can we bridgeθ this i
gap?
δ ad = θ T ω
= −Γ B Pe ω
r h H, Vx i
T
pi i u
T

i θ

3 Adaptive Parameters δ nom

H cmd , Vcmd
δ
δ nom = K x x
8 Adaptive
x Parameters 
θ = −Γ i B pi Pe
i uω
δ ad = θ T ω
T T

δ ad
i

θ
ucmd x
x (coupling removed)
δ θ ω = T

MH-96hController (1957) Provably Correct AFCS (2007)

ad

3 Adaptive Parameters
3 Adaptive Parameters
x (α, β, integral
(PCstates
Adaptive law replaced
removed;
eu replaced with e) MH-96 logic)
PC Adaptive law is key!
Anti-windup solutions: 1. Magnitude Saturation
Input u R (u ) P
Saturation Plant

Plant x p = Ap x p + B p Λ R(u ) ; unknown Ap , Λ

Magnitude Constraints

 u1,max sat1 (u1 )

  ui if | ui |≤ ui ,max
R (u ) =   sati (ui ) = 
u ui ,max if | ui |> ui ,max
 m ,max sat m (um )

Problem: Ensure stability and performance in the presence of magnitude limits

27
An Adaptive Solution*
Saturation Error
Nominal ∆u = sat (u ) − u
r Cont. u sat (u )
P
Adaptive
Law
∆u
KA xp

Plant Control Law

x p = Ap x p + B p Λsat (u )
u = Kx p + r
Reference Adaptive Law Replace e with eu
xm = Am xm + Br r K = −Γdiag ( sign(λ )) B Peu x p
T T

Generate a signal eu 
λˆ = Γλ diag (∆u ) B T Peu
e = x p − xm where AmT P + PAm = −Q, Q > 0
eu = e − e∆
e∆ = Am e∆ + Bdiag (λˆ )∆u Semi-global stability can be shown.
28
* S. Karason and A.M. Annaswamy, IEEE Trans. Autom. Control, 1994.
 Results: HSV with Flexible Effects
Multiplicative uncertainty Initial conditions Reference commands
Mach 8
85,000 ft

Baseline: LQR + Integral Action

Velocity command error Equivalence ratio in scramjet engine

29
2. Adaptive Reset Control
Controller
+ Integral u sat (u )
Action
P

Resetting

Resetting strategy based on:

- Imposed saturation limit on
error integral state
sat ( xc ) = f c ( xc , u (t ))

- State resetting in
error integral state
xct r(tj rj+ ) = sat ( xc (t rj ))
where trj is the time instance
at which (i) xc (trj ) = 0 (ii) u (trj ) ≥ u0

tracking performance 30
Adaptive Reset Control: Control Architecture

Plant with Integral Action Control Law

 x p   Ap 0  x p   B p  0 uc = K p x p + K c sat ( xc )
= +
 x   H 0  x   0  Λ sat (u ) + − I  r
 c 
   c   Adaptive Law
x
K = −Γdiag ( sign(λ )) B T Peu xT
A x B Br

Reference Model

xm = Am xm + Br r λˆ = Γλ diag (∆u + K c ∆c) B T Peu
Error AmT P + PAm = −Q, Q > 0
e = x − xm
Resetting xc (t rj+ ) = sat ( xc (t rj ))
eu = e − e∆
when (i) xc (t rj ) = 0 (ii) u (t rj ) ≥ u0 31
e∆ = Am e∆ + Bdiag (Λ
ˆ )(∆u + K ∆c)
c
Semi-global stability can be shown.
Adaptive Reset Control: A First-order Example
States

 Reference Input - step

 Error integral state is augmented
 Input magnitude constrained
 Comparison of
- Adaptive Reset Controller
- Adaptive Controller with KA Inputs
- Basic Adaptive Controller
- Nominal Controller

Magnitude Saturation Limits

Error Integral State

Resetting occurs
(twice)
32
A Transport Aircraft (uncertainty: CG shift)
States
Nominal Controller Adaptive Reset Controller

With CG change along x-axis, we observe strong oscillations with

nominal controller which disappears with adaptive reset controller.
33
A Transport Aircraft (uncertainty: CG shift)
Control Inputs
Nominal Controller Adaptive Reset Controller

System with adaptive reset controller leaves saturation

earlier due to the resetting strategy. 34
 3. Systems with Input Rate Saturation
Rate Saturation:

Input u R (u ) 1 ua
s P
Saturation Plant

Plant x p = Ap x p + B p Λua
Input Constraints u ≤ um 0 , u ≤ ur 0

Problem: Ensure stability and performance in the presence of rate and magnitude limits

Rate & Magnitude Saturation Model:

um sat (um ) + ur sat (ur ) ua
1 1
- τ s

35
An Adaptive Solution

Magnitude and Rate-limit Model

Nominal sat (um ) sat (ur )

r Controller um + 1 ur 1 ua P
- τ s
xp
Adaptive
Controller
∆um
AW ∆ur

Augmented System xp  Control Law

x =  xc  u = K p x p + K c xc
ua 
Adaptive Law
Error e = x − xm K = −ΓB T Peu xT
eu = e − e∆ 
1 Aˆ ∆ = −ΓA Peu xT
e∆ = Am e∆ + Aˆ ∆ x + B2 ( ∆um + ∆ur ) 36
τ where Am P + PAm = −Q, Q > 0
T
Simulation Results
First Order Plant

 Reference Input – doublets States

 Error integral state is augmented xc = hx p − r
 Plant a p = 0.6
bp = 1
Model
am = −1

 Saturation Limits - Magnitude umax = 5

Rate umax = 10
Inputs
 Comparison of
- Nominal Controller
- Basic Adaptive Controller
- Adaptive Controller with KA modification
- Suggested Adaptive Controller

Control Signal
(Suggested Adaptive Controller)
rate saturation magnitude saturation
37
Optimization of Non-adaptive Components
Reference
Model
Plant

Nominal
Controller
Saturation
Adaptive
Controller mux

• Reference Model: Structure and parameters

• Nominal controller
• Adaptation gains
38
 Optimization of Control Parameters

• Uncertainty / Failure p

• Set of closed-loop requirements gi(p,d)<0

• Controller parameter d

• nominal point p0
One-dimensional Setting:
• critical requirement g1; g2
S1 • critical point p*
g1 g2

S • safety domain S

• Determine d so
that the safety
p* p0 p domain is
p*
maximized

39
S2
 Margins of Adaptive Systems

Nominal Time Delay

r Cont. u
e −τs P
Adaptive
Law
xp

Problem Statement: Evaluate the stability margins of adaptive systems

- What is the worst time delay which adaptive systems can tolerate?

• Extend the notions of gain and phase margins to

adaptive systems
 Delay Margins

Derive the lower bounds of delay margins based on Pade approximations.

• System:
- Plant: x (t ) = Aλ x(t ) + bλu (t − τ ) y (t ) = cT x(t )

- Control Law: u (t ) = (k T + θ T (t )) x(t )

- Adaptive law: θ(t ) = −γx(t )eT (t ) Pb p sign(λ ) − σθ (t )

Theorem. The system, in the presence of (n, n) Pade approximation, with the
controller based on the adaptive law, there existτ m > 0 and ymax > 0 such that θ (t ) , x(t )
have bounded trajectories for all t > t0 if
i. 0 < τ < τ m
ii. || y (0) ||< ymax
τ m is defined as a delay margin.
Adaptive Control for an Embedded System

• Explicit accommodation of network parameters: partial

knowledge of the time delay
• Design an adaptive controller which accommodates the
network structure and adapts to uncertainties
Adaptive Control of a NCS

• Dedicated Network Control Systems with Shared Resources

– Use as much information from the schedule as possible to design a
more efficient controller

– The total delay consists of two parts: a known part due to the
schedule and an unknown part due to communication constraints
Adaptive Control of a NCS

• Linear system

• Sampled system

• Control input

• Update laws

• Stability proof
 Results
Fixed Controller
– Linear plant

– At , the sampling
rate changes from 1ms
to 10ms
– This can happen due to
power consumption
constraints or changes in
the environment Adaptive Controller
– The adaptive controller
follows the reference
input whereas the fixed
controller cannot deal
with this change in the
protocol
Validation

• Aerospace, Automotive, and Propulsion

Systems
– High Fidelity Simulations
– Hardware-in-the-loop (HIL) Tests
– Stand-alone Hardware Tests
 Quadrotors

• Adaptive controller  maintains stability in

the presence of ≈40% loss of thrust

• Standard, linear controller  unable to stabilize

47
Multiple Quadrotors: Adaptive Formation Flight

Nominal Controllers Adaptive Controllers

Comparison of Adaptive methods

• Quadrotor flight control system has ~40

ms delay
• Can we use the modifications to MRAC
discussed above to improve
performance?
Flexible & Reconfigurable UAV

KS , KD

J ′′

“ParaFlex”
Manipulation with UAV
“QuadPush, QuadPull,
QuadGrip, QuadFlex”
F τ
F sin (θ )

PUSH

Contact

Single UAV Adaptive

Push Control
 A Generic Transport aircraft Model
GTM GTM: Simulink Flight Simulator
•5.5% dynamically scaled transport • Experimentally obtained extended aero data set
Vehicle at NASA for high AOA and spins
• Sensor noise and sensor bias from flight data
• Telemetry uplink and downlink time delays
• Actuator dynamics with rate and position limits
• Sensor dynamics along with ADC and DAC
latencies and quantization
AoA Command
Roll Command

Time
 Adaptive Control of the GTM
Baseline

AoA
•LQR-PI
Baseline

Adaptive Controller

Roll
•Saturation
Time
•Anti Windup
•Nonlinear Engine Model
•20ms Analog Delay in reference
•60ms Digital Delay in reference
Scenario
Analog time delay 60 ms (nominal 20ms)
Digital time delay 105 ms (nominal 60ms)
Left aileron Locked -10 deg
Adaptive

Left inboard elevator Locked 0 deg

Bottom rudder Locked 0 deg
Top rudder 25% effective
Left outboard elevator 50% effective
All right elevators 25% effective
Center of gravity shift -45% MAC
 Summary: Foundations, Verification & Validation
Stand alone hardware tests
Validation

Hardware-in-the-Loop tests
High Fidelity simulation studies
Integration with Baseline

Saturation & Resetting

adaptive Components
Optimization of Non-

Embedded adaptive
Verification Tools

Anti-windup:

Margins
Designs

control
Unmatched Uncertainties
Foundation

Nonlinearities: Large delays:

Dynamics, parameterization Posicast adaptation, Composite adaptive laws
Robust parameter adaptation
Model Reference Adaptive Control
For further details, contact:
Actuation Uncertainties: Amith Somanath
Magnitude and Rate Limits: Megumi Matsutani
Higher Order Dynamics: Travis Gibson
Time-delays: Zachary Dydek
Hardware-in-the-Loop: Manohar Srikanth
Coordination and Co-design: Harald Voit
Application to Energy Systems: Arman Kiani

Collaborators: Sponsors:
Eugene Lavretsky (Boeing) Boeing
Ilya Kolmanovsky (U Michigan) Ford
Jean-Jacques Slotine (MIT) NASA (Aeronautics & Aviation Safety)
Jonathan How (MIT) TU München Institute for Advanced Study
Nick Roy (MIT)