Control Principles for Engineered Systems 5SMC0

Adaptive Control
Lecture 1

Zhiyong Sun

Control Systems Group

Department of Electrical Engineering
Eindhoven University of Technology

1
Where are we now (in the map of control theory)

2
Adaptive Control
Lecture 1
Introduction to Adaptive Control
& Model Reference Adaptive Control

Main reference I Main reference II

Chapter 5 Chapter 9
3
3
Outline

• Introduction and history of adaptive control

• Signal convergence, Barbalat’s lemma
• Adaptive control structure
• Model reference adaptive control

4
4
Introduction and historical review

5
5
What is adaptive control?

• Richard E. Bellman: “…Even less is known about the system and

the controller has to learn to improve its performance through the
observation of the outputs of the system as it evolves.”

• Karl Johan Åström: “An adaptive controller is a controller with

adjustable parameters and a mechanism of adjusting the
parameters.”

6 Richard E. Bellman Karl Johan Åström 6

Historic review of adaptive control

• Adaptive systems
• Caught the imagination in the 1950-60s
• First use in flight control and design of autopilot operations
• Hypersonic program - highly successful

• Progress of adaptive control

• First in use in 1950s-1960s (flight control, MIT rules, by NASA)
• First wave in 1970s (theoretical analysis).
• Stability and convergence guarantee, 1970s-1980s
• Robustness issues and robust adaptive control, 1980s-1990s
• Intelligent adaptive control, 1990s-2000s.
• Data-driven adaptive control and learning-based control, from
2000s.

7 7
History of adaptive control: flight application perspective
X-15
(NASA/USAF/ US Navy)

Intelligent adaptive
control (NN, Fuzzy
logic etc. )

Gain-scheduling
Current-generation
MIT rule
(NASA) robust adaptive control
F-8

1950 1960 1970 1980 1990 2000 2010 ??

Certifiable
Adaptive control
Robustness
Robust modification
Model-reference
adaptive control
Lyapunov theory
GTM T2 (NASA)

(NASA/Boeing)
8 F-15 8
Adaptive control structure

Benefits of adaptive control

• It provides consistent performance in the presence of modeling uncertainties,
large parameter variations and unknown parameters.
• It does not need accurate system model.

An adaptive control system can be thought of as having two loops.

• One loop is a normal feedback loop with the process and the controller.
• The other loop is the parameter adjustment loop.

In the presence of uncertainties, using prior and on-line information, the

controller adapts itself.
9 9
Motivating example I

10 10
Motivating example I

11 11
Motivating example II

12 12
Motivating example II

13 13
Motivating example II

14 14
Signal convergence & Barbalat's Lemma

15
15
Recap: signal norms

16 16
Recap: signal (function) space

17 17
Commonly-used signal spaces in adaptive control

In proving stability for adaptive systems, we are initially concerned

with showing boundedness of signals, such as the error 𝑒(𝑡).

• If this error signal 𝑒(𝑡) is bounded for all time, we write this formally as

implying that the norm of 𝑒(𝑡) exists and is finite.

• Similarly, if the norm of a signal 𝑒(𝑡) exists and is finite, we write

• If a signal 𝑒(𝑡) is bounded for all time and has a finite norm, we write

18 18
Recap: signal (function) space

Online quiz: determine the function space(s) that the following signals belong to

1
(I) 𝑓 𝑡 = 𝑡 , 𝑡 ≥ 1 (the limits of integral are [1, \infty) )
(a) (b) (c)

(II) 𝑓 𝑡 = sin 𝑡 , 𝑡≥0

(a) (b) (c)

(III) 𝑓 𝑡 = 𝑒 −𝑡 , 𝑡 ≥ 0
(a) (b) (c)

19 19
Signal convergence

In general, the fact that derivative tends to zero does not

imply that the function has a limit.

Also, the converse is not true: the fact that a function has
a limit does not imply that its derivative tends to zero.

Counterexamples:

20 20
Barbalat's lemma for signal convergence

Barbalat's lemma:

Barbalat's lemma is one of the key lemmas to prove signal convergence

in adaptive control.

A time-dependent function tends to a limit and if its time derivative is

uniformly continuous, then the derivative tends to zero.

21 21
Barbalat's lemma for signal convergence

Why is the condition of uniform continuous function 𝑓(𝑡)?

Consider the following function described by a sequence of isosceles triangles of
1
base length 2 and height equal to 1 centered at 𝑛 where 𝑛 = 1, 2, . . . ∞ as
𝑛
shown in the figure below:

1
This function is continuous but not uniformly continuous.

but does not exist!

22 22
Application of Barbalat’s lemma
Consider the following scalar system with adaptive parameter updating

A time-varying function, for

external (reference) input
Lyapunov function

Its derivative

Then both 𝑒(𝑡) and 𝜃(𝑡) are uniformly bounded.

Its second-order derivative

By Barbalat’s lemma, the above in turn indicates that 𝑒(𝑡) tends to zero.

23 23
Model Reference Adaptive Control (MRAC)

24
24
Uncertainty classification in uncertain plant
Structured uncertainty (aka parametric uncertainty): uncertain parameters but known
functional characteristics
• Exp: A linear spring-mass-damper system with an uncertain spring constant

Unstructured uncertainty: neither parameters or functional characteristics are certain

• Exp: A spring-mass-damper system with an uncertain spring characteristic

Unmodeled dynamics: uncertain system internal or external dynamics that are not included
in a plant model
• Exp: A spring-mass-damper system with an uncertain spring characteristic

Matched uncertainty: structured uncertainty that can be matched by the control input

a parametric matched uncertainty

Unmatched
25 uncertainty 25
Model reference adaptive control (MRAC)

Direct adaptive control

• Estimate the controller parameters

• The stable error dynamics and adaptive laws are derived using the structure
of the control signal

Indirect adaptive control

• Estimate the system parameters

• The stable error dynamics and adaptive laws are derived independent of the
control signal
26 • The control signal is synthesized using the estimated system parameters 26
(certainty equivalence)
Direct MRAC for scalar SISO systems

27
27
Model reference control for scalar SISO systems
Consider a first-order nonlinear SISO system

structured matched uncertainty:

an unknown constant vector:

a vector of known bounded
basis functions:

A reference model is specified as

a uniformly bounded model

reference input

is a uniformly bounded model reference signal.

28 28
Direct MRAC for scalar SISO systems: case of unknown parameters
Consider a first-order nonlinear SISO system (suppose 𝑎 and 𝑏 are unknown,
but sign of 𝑏 is known )

A reference model

An ideal controller that perfectly cancels out the uncertainty and enables 𝑥(𝑡) to
follow 𝑥𝑚(𝑡) is

Model matching conditions:

Adaptive control law

29
Direct MRAC for scalar SISO systems: case of unknown parameters
Consider a first-order nonlinear SISO system (suppose 𝑎 and 𝑏 are unknown,
but sign of 𝑏 is known )

A reference model

Model matching conditions:

Adaptive control law:

Gain and parameter

adaptation errors:

Let be the tracking error

30
Direct MRAC for scalar SISO systems: case of unknown parameters
Consider a first-order nonlinear SISO system (suppose 𝑎 and 𝑏 are unknown,
but sign of 𝑏 is known )

A reference model

The tracking error dynamics

Lyapunov function

Its time derivative

31
Direct MRAC for scalar SISO systems: case of unknown parameters
Consider a first-order nonlinear SISO system (suppose 𝑎 and 𝑏 are unknown,
but sign of 𝑏 is known )
to track a reference model

Time derivative of the Lyapunov function

To make , we design

The adaptive laws Adaptation (or learning) rates

32
Direct MRAC for scalar SISO systems: case of unknown parameters
Consider a first-order nonlinear SISO system (suppose 𝑎 and 𝑏 are unknown,
but sign of 𝑏 is known )
to track a reference model

Lyapunov function
Time derivative of the Lyapunov function
are bounded ( )

is bounded ( )

The Lyapunov function has a limit as .

The second derivative of the Lyapunov function

is bounded
is uniformly continuous

By Barbalat’s lemma as
33
Summary: direct MRAC for scalar SISO systems
Consider a first-order nonlinear SISO system (suppose 𝑎 and 𝑏 are unknown,
but sign of 𝑏 is known )
to track a reference model

The tracking error dynamics

Lyapunov function

The adaptive laws

By Barbalat’s lemma as

• The tracking error 𝑒(𝑡) is asymptotically stable.

• Adaptive signals (adaptive gains and parameter estimations) are bounded.

34
Direct MRAC for scalar SISO systems: case of known parameters

Consider a first-order nonlinear SISO system (suppose 𝑎 and 𝑏 are known)

A reference model

The model matching conditions

Adaptive controller

The tracking error equation

The adaptive law can be found

35 35
Direct MRAC for scalar SISO systems: case of known parameters
The tracking error equation
The adaptive law can be found

Lyapunov function

Its time derivative

36

The Barbalat’s lemma implies that

• The tracking error 𝑒(𝑡) is asymptotically stable.

• The parameter estimation is bounded (for convergence to true values,
additional conditions are required, such as the PE—persistent excitation 36
condition).
Direct MRAC for scalar SISO systems: case of known parameters

Consider a first-order nonlinear SISO system (suppose 𝑎 and 𝑏 are known)

A reference model

37 37
Indirect MRAC for scalar SISO systems

38
38
Indirect adaptive control (MRAC): certainty equivalence

Indirect adaptive control

• Estimate the system parameters

• The control signal is synthesized using the estimated system parameters

Certainty equivalence
• Step 1: Algebraic part: find a solution to the problem when parameters are
known.
• Step 2: Analytic part: replace the unknown parameters by their estimates.
Ensure stable and convergent behavior.
• The use of the parameter estimates in place of the true parameters is known
39 as the certainty equivalence principle. 39
Indirect MRAC for scalar SISO systems
Consider a first-order nonlinear SISO system (suppose 𝑎 and 𝑏 are unknown,
but sign of 𝑏 is known )
to track a reference model

The model matching conditions

Indirect adaptive control law

The tracking error equation

40
Indirect MRAC for scalar SISO systems
Consider a first-order nonlinear SISO system (suppose 𝑎 and 𝑏 are unknown,
but sign of 𝑏 is known )
to track a reference model

The tracking error equation

Lyapunov function candidate

Adaptive laws

41
Indirect MRAC for scalar SISO systems
Consider a first-order nonlinear SISO system (suppose 𝑎 and 𝑏 are unknown,
but sign of 𝑏 is known )
to track a reference model

Lyapunov function
Time derivative of the Lyapunov function
are bounded ( )

is bounded ( )

The Lyapunov function has a limit as .

The second derivative of the Lyapunov function

is bounded
is uniformly continuous
By Barbalat’s lemma
• The tracking error 𝑒(𝑡) is asymptotically stable.
Modified indirect MRAC for scalar SISO systems
Consider a first-order nonlinear SISO system (suppose 𝑎 and 𝑏 are unknown,
but sign of 𝑏 is known )
to track a reference model

The model matching conditions

Indirect adaptive control law

Adaptive laws

It may occur that , while the adaptive control law “blows up”.

Suppose a lower bound on b is known

A modified adaptive law via the projection method

෠ will always be greater

|𝑏|
than or equal to 𝑏0

By Barbalat’s lemma, 43
Direct MRAC for MIMO systems

44
44
Direct MRAC for MIMO systems

Consider a nonlinear MIMO system (assume that A and B are known)

A reference model

The model matching conditions

Adaptive controller

The tracking error equation

The adaptive law can be found

45 45
Direct MRAC for MIMO systems
The tracking error equation
The adaptive law
Positive definite matrix (for
Lyapunov function adaptation rate)

with

Its time derivative

46

The adaptive law

The Barbalat’s lemma implies that

• The tracking error 𝑒(𝑡) is asymptotically stable.

• Parameter estimation is bounded (if with PE condition, then convergence to
true value is guaranteed.) 46
Procedure of adaptive control design

• Specify system uncertainties (unknown parameters, matched

uncertainties etc.);

• Choose direct/indirect adaptive control (adjust gains, or estimate

parameters);

• Inverse Lyapunov approach: construct a Lyapunov function (often in a

quadratic form), and make its first derivative negative semi-definite;

• Apply Barbalat’s lemma for error signal convergence and estimation

boundedness.

47
Summary/take home messages

• History of adaptive control

• Barbalat’s lemma, signal convergence

• Direct/indirect adaptive control

• Model reference adaptive control (MRAC), for SISO/MIMO

system

48
The next lecture

• Robust adaptive control.

49
Appendix: vector norms

• Vector norms for a vector

• Commonly-used vector norms:

50
Appendix: matrix norms

• The induced p-norm of a matrix A is defined as:

• Commonly-used induced matrix norms:

• Norm inequalities

51
Appendix: positive definite matrix and quadratic Lyapunov function

• Consider a symmetric positive definite matrix P:

52
Appendix: matrix equalities

• Matrix trace

• Matrix derivative

53