Indian Institute of Technology Bombay

Department of Aerospace Engineering

AE 666 Adaptive and Learning Control Systems
Spring 2024-2025

Tutorial 4 Solution April 15, 2025

1. Given a convex function f (x) : Rn →

− R and a constant δ > 0 prove the following:
(a) The subset Ωδ = {θ ∈ Rn |f (θ) ≤ δ} is convex.
(b) Now suppose f (x) : Rn → − R be a differentiable convex function. Let θ⋆ ∈ Ωδ and assume
that f (θ ) < δ, that is, θ∗ is an interior point (i.e not on the boundary) of Ωδ . Also, let
∗

θ ∈ Ωδ and assume that f (θ) = δ, that is, θ lies on the boundary of Ωδ . Then show that the
following inequality holds
(θ∗ − θ)T ∇f (θ) ≤ 0,
∂f (θ) T
 
∂f (θ)
where ∇f (θ) = ,..., ∈ Rn is the gradient vector of f evaluated at θ.
∂θ1 ∂θn

Solution: For the given convex function f (x) : Rn →

− R and the constant δ > 0.
(a) Let θ1 , θ2 ∈ Ωδ . Then, f (θ1 ) ≤ δ and f (θ2 ) ≤ δ. Since f (x) is convex, then for any 0 ≤ λ ≤ 1,

f (λθ1 + (1 − λ)θ2 ) ≤ λf (θ1 ) + (1 − λ)f (θ2 )

≤ λδ + (1 − λ)δ
≤δ

Therefore, f (θ) ≤ δ and consequently θ ∈ Ωδ .

(b) Since f (x) is convex, then

f (λθ∗ + (1 − λ)θ) ≤ λf (θ∗ ) + (1 − λ)f (θ)

or

f (θ + λ(θ∗ − θ)) ≤ f (θ) + λ(f (θ∗ ) − f (θ))

Consequently, for any nonzero 0 < λ ≤ 1,

f (θ + λ(θ∗ − θ)) − f (θ)

≤ f (θ∗ ) − f (θ)
λ
≤0

Taking the limit as λ → 0 gives the inequality.

AE 666 Tutorial 4 Solution April 15, 2025

2. Let f (θ) be a convex continuously differentiable map from Rn → R. Consider the n-dimensional
dynamics
θ̇ = Proj(θ, y),
where θ ∈ Rn is the system state and y ∈ Rn is a time-varying piecewise continuous vector. Prove
that starting from any initial condition θ(0) = θ0 within the set

Ω0 = {θ ∈ Rn |f (θ) ≤ 0},

the system trajectory θ(t) will remain in the set

Ω1 = {θ ∈ Rn |f (θ) ≤ 1}.

for all t ≥ 0. For any symmetric positive-definite matrix Γ ∈ Rn×n ,

 T
Γy − Γ∇f (θ)(∇f (θ)) Γyf (θ), if f (θ) > 0 and y T ∇f (θ) > 0
Proj(θ, Γy) = (∇f (θ))T Γ∇f (θ) .
Γy, otherwise


Solution: Existence and uniqueness of the system solutions are provided by the fact that the Projection
Operator is locally Lipchitz in θ, while the system external input y is piecewise continuous in time.

f˙(θ) = (∇f (θ))T θ̇ = (∇f (θ))T Proj(θ, y)

 T T
(∇f (θ))T y − (∇f (θ)) Γ∇f (θ)(∇f (θ)) yf (θ), if f (θ) > 0 and y T ∇f (θ) > 0
f˙(θ) = (∇f (θ))T Γ∇f (θ)
 T
(∇f (θ)) y, if not

we obtain
(∇f (θ))T Γ∇f (θ)(∇f (θ))T
yf (θ) = (∇f (θ))T yf (θ)
(∇f (θ))T Γ∇f (θ)
(
(∇f (θ))T y(1 − f (θ)), if f (θ) > 0 and y T ∇f (θ) > 0
f˙(θ) =
(∇f (θ))T y, if not



> 0, if 0 < f (θ) < 1 and y T ∇f (θ) > 0
f (θ) = 1 and y T ∇f (θ) > 0

= 0, if
f˙(θ) =
> 0,

 if f (θ) ≤ 0 and y T ∇f (θ) > 0
f (θ) ≤ 0 and y T ∇f (θ) ≤ 0

≤ 0, if
The first and second condition implies that if f (θ) > 0 then f (θ) monotonically increases in time for
all t ≥ 0, but will never exceed 1. For f (θ) > 0 with y T ∇f (θ), the trajectory might come out of the
set Ω0 but due to first and second condition it remains in Ω1 . Also, the fourth condition implies that
the trajectory remains in Ω0 . Therefore for any initial condition θ0 such that f (θ0 ) ≤ 0, f (θ(t)) ≤ 1 for
all t ≥ 0.

2
AE 666 Tutorial 4 Solution April 15, 2025

3. Consider the following scalar system

ẋ (t) = −x (t) + ωu (t) , x (t0 ) = x0 ,

where ω is an unknown parameter with known sign and lower and upper bounds. Design an L1
adaptive controller and simulate the results for ω = 1, x0 = 0, ŵ (t0 ) = 0.5, take control gain as
1
5, filter C (s) = , and choose adaptation gain as 105 .
s (0.05s + 1)

Solution: Let the predictor model for the given scalar system be

x̂˙ (t) = −x̂ (t) + ω̂ (t) u (t) , x̂ (t0 ) = x0 .

The control law is

u (s) = kC (s) (kr r (s) − η̂ (s)) ,
where η̂ (s) and r (s) are the Laplace transforms of η̂ (t) = ω̂ (t) u (t) and r (t), respectively; kr = 1; and
k > 0 and C (s) are a feedback gain and a strictly proper transfer function. The adaptation law to
update ω̂ (t)
ω̂˙ (t) = γProj (ω̂ (t) , −x̃ (t) u (t)) .

1 1

0.5 0.5

0 0
0 2 4 6 8 10 0 2 4 6 8 10

1
Figure 1: ω = 1, x0 = 0, ŵ (t0 ) = 0.5, k = 5, C (s) = , γ = 105 .
s (0.05s + 1)

The step size for the numerical simulation is chosen to be 10 µs.

4. Let a first-order ordinary differential equation represent an actuation device

τ u̇ = ucmd − u,

where ucmd is the actuator-commanded position, u is the actuator-achieved position, and τ is the
actuator time constant.
(a) Design an actuator model using the Projection Operator to ensure the actuator output
satisfies the constraint
|u| ≤ umax ,
where umax is the actuator’s position limit.

3
AE 666 Tutorial 4 Solution April 15, 2025

(b) Observe the evolution of the actuator-achieved position (u) using numerical simulation.
Take τ = 0.5, umax = ±1, and select ucmd = sin t.

Solution: Let us consider the following convex function

(1 + ϵ) u2 − u2max
f (u) = , ϵ ∈ (0, 1) .
ϵu2max
The actuator model using Projection Operator

 cmd − u (1 − f (u)) , if f (u) > 0 and (ucmd − u) u > 0

u

u̇ = ucmdτ − u .
 , otherwise
τ

1 2

0.5 1

0 0

-0.5 -1

-1 -2
0 2 4 6 8 10 0 2 4 6 8 10

Figure 2: τ = 0.5, umax = ±1, ucmd = sin t and 2 sin t.

1 2

0.5 1

0 0

-0.5 -1

-1 -2
0 2 4 6 8 10 0 2 4 6 8 10

Figure 3: τ = 0.05, umax = ±1, ucmd = sin t and 2 sin t.

The step size for the numerical simulation is chosen to be 1 ms and ϵ = 0.8.

5. Consider a system where the output is governed by unknown dynamics that can be represented
in a linearly parameterized form
y(t) = w∗T ϕ (x (t)) ,
where y(t) denotes the measured output, x : R+ → Rn , and ϕ : Rn → Rp , n, p ∈ N, are known
functions, while w∗ ∈ Rp is an unknown constant parameter vector.

4
AE 666 Tutorial 4 Solution April 15, 2025

(a) Design a gradient-based adaptive law to update the estimate of the unknown vector w∗ and
modify it using the concurrent learning method.
(b) Derive the parameter error dynamics and state the conditions on the stored data set required
for this concurrent learning approach to guarantee ideal convergence of the parameter
estimate.

Solution: The unknown system dynamics is given by

y(t) = w∗T ϕ(x(t)). (1)

Let w(t) ∈ Rp denote the online estimate of the ideal weights w∗ , hence, the online estimate of y can
be given by
ν(t) = wT (t)ϕ(x(t)), (2)
where w is the online parameter estimate of w∗ . The estimation error in the output and the parameter
estimate are denoted by ϵ and w̃, respectively, and are defined as ϵ(t) = ν(t) − y(t) and w̃ = w − w∗ ,
respectively, hence,
ϵ(t) = w̃T (t)ϕ(x(t)).

The following gradient-based adaptive law

ẇ(t) = −Γϕ(x(t))ϵ(t),

ensures that the output estimation error goes to the origin, where Γ is a symmetric positive definite
adaptation gain matrix.
The gradient-based adaptive law with concurrent learning term is
p
X
ẇ(t) = −Γϕ(x(t))ϵ(t) − Γϕ(xj )ϵj , (3)
j=1

where j ∈ {1, 2, . . . , p} denote the index of a stored data point xj , ϕ(xj ) denote the regressor vector
evaluated at point xj , and ϵj = w̃T ϕ(xj ).
Assuming Γ to be an identity matrix, without any loss of generality, the parameter error dynamics is
p
X
˙
w̃(t) = −ϕ(x(t))ϵ(t) − ϕ(xj )ϵj
j=1

p
X
T
= −ϕ(x(t))ϕ (x(t))w̃(t) − ϕ(xj )ϕT (xj )w̃(t)
j=1
 
p
X
= − ϕ(x(t))ϕT (x(t)) + ϕ(xj )ϕT (xj ) w̃(t). (4)
j=1

This is a linear time-varying equation in w̃.

5
AE 666 Tutorial 4 Solution April 15, 2025

1
Let V (w̃) = w̃T (t)w̃(t) be a Lyapunov function candidate. There exist α > 0 and β > 0 such that
2
α∥w̃∥2 ≤ V (w̃) ≤ β∥w̃∥2 . The time derivative of V along the trajectories of
 
p
X
V̇ (w̃) = −w̃T (t) ϕ(x(t))ϕT (x(t)) + ϕ(xj )ϕT (xj ) w̃(t). (5)
j=1

Pp
Let Ω(t) = ϕ(x(t))ϕT (x(t)) + j=1 ϕ(xj )ϕ
T
(xj ), then

V̇ (w̃) = −w̃T (t)Ω(t)w̃(t),

for Ω(t) to be a symmetric positive definite matrix, the following condition is required.
The recorded data has as many linearly independent elements as the dimension of ϕ(x(t)). That is, if

Z = [ϕ(x1 ), . . . , ϕ(xp )]

then rank(Z) = p.
If the stored data points satisfy the above condition,

V̇ (w̃) ≤ −λmin (Ω) w̃T (t)w̃(t) ≤ −λmin (Ω) ∥w̃∥2 ,

inferring w̃ ≡ 0 is globally exponentially stable when using the concurrent learning based adaptation
law.

6. Let the following algebraic system represents a system,

 
1 
0.1
y(t) = w∗T ϕ (θ) , ϕ(θ) =  − θ− π 2  , ∗
w = .
2 0.6
e

(a) Using the gradient-based adaptive law designed in Q.5, observe the evolution of the parameter
estimate using numerical simulations. Take any suitable bounded function θ (t).
(b) Use the concurrent learning based adaptation law designed in Q.5, and compare the evolution
of parameter estimate using numerical simulation. Comment on which method provides
ideal parameter convergence.

Solution:

6
AE 666 Tutorial 4 Solution April 15, 2025

Figure 4: Time evolution of the output and its estimate.

Figure 5: Comparison of weight convergence