Proceedings o l the 31rt Conference

on Lbdrlon and Control

Tucson. Arhona *December 1962
WP9 - 18: OO
State-Space Solutions to Discrete-Time and
Sampled-Data 3-12 Control Problems1
Tongwen Chen
Dept. of Elect. & Comp. Engg.
University of Calgary
Calgary, Al berta
Canada T2N 1N4
Abstract
This paper gives a complete state-space derivation of the
discrete-time "2-optimal controller. This derivation can be
extended to treat a sampled-data 'HZ control problem, resulting
in a new direct solution to the sampled-data problem.
1 Introduction
A recent trend in synthesizing sampled-data systems is to use the
more natural continuous-time performance measures. This brought
solutions to several new "2-optimal sampled-data control problems
(6, 4, 131, each reducing to an 7 1 2 problem in discrete time.
Discrete-time ?i 2 (LQG) theory was developed in the 197O's, see,
e.g., [2, 8, 14, 15, I , 161. As in the continuous-time case, the discrete
optimal controller is closely related to the solutions of two Riccati
equations. In [9], the solution to a continuous-time "2-optimal con-
trol problem was rederived using the state-space approach. This gives
a clean treatment of the problem and provides compact formulas for
the optimal controller. Since complete. general formulas for the dis-
crete optimal controller are not readily available in the literature, we
ask the question here, can a state-space treatment be accomplished
for discrete-time H2 problems?
The goal in this paper is twofold: to present a state-space solution
to the discrete " 2 control problem and to give direct formulas for an
"2-optimal sampled-data control problem with state feedback and
disturbance feedforward using the powerful lifting technique [17, 19,
The organization of the paper is as follows. In the next section
we collect and some preliminary results on Rjccati equations; the
presentation follows closely that in [9] in continuous time. Section 3
treats the discrete-time "2-optimal control, first via state feedback
and disturbance feedforward and then vi a dynamic output feedback.
Section 4 presents new direct formulas for a sampled-data 312 problem
using state measurement. In Section 5 weapply the optimal sampled-
data control in Section 4 to a two-motor system and compare with the
optimal analog control. Most proofs are omitted and can be found in
The notation in this paper is quite standard: C is the complex
plane, ' D C C is the open unit disk, and 82, is the boundary of
'D, namely, the unit circle. Also, 2 is the set of all integers and 2+
(2-) is the nonnegative (negative) subset of 2. The space l z( Z+) , or
simply l z , consists of all square-summable sequences, perhaps vector-
valued, defined on 2+. Similarly for ez(Z) and &(Z-). The discrete-
time frequency-domain space ' H*(D), or simply "2, is the Hardy space
defined on V. We use R'H2 for the real-rational subspace of 712. In
discrete time, we use A-transforms instead of z-transforms, where
X =z-l. Finally, g " ( A) stands for the transposed matrix g ( l / X) ' .
3, 51.
(71.
'The work was supported by the Natural Sciences and Engineering Research
Council of Canada.
2
Bruce A. Francis
Dept. of Elect. Engg.
University of Toronto
Toronto, Ontari o
Canada M5S 1A4
Riccati Equation
It is well-known that Rjccati equations play an important role in the
' H2 optimization problem. The solution of a Riccati equation can
be obtained vi a the stable eigenspace of the associated symplectic
matrix if the state transition matrix of the plant is nonsingular. If
this matrix is singular, as is the case when the plant has a time delay,
then the symplectic matrix is not defined; but we can use the stable
generalized eigenspace of a certain matrix pair [16].
Let A, Q, R be real n x n matrices with Q and R symmetric.
Define the ordered pair of 2n x 2n matrices
A pair of matrices of this form is called a symplectic pair. (This
definition is not the most general one.) Note that if A is nonsingular,
then H;' HI is a symplectic matrix.
Introduce the 2n x 2n matrix
J :=[ - I ]
I O '
It is easily verified that H I J H I =H2 J H; . Thus the generalized
eigenvalues (including those at infinity) for the matrix pair H (i.e.,
those numbers X satisfying Hl x =XHz z for some nonzero z) are
symmetric about the unit circle, i.e., X is a generalized eigenvalue iff
l / X is [16].
Now we assume H has no generalized eigenvalues on 8V. Then
it must have n inside and n outside. Thus the two generalized
eigenspaces Xt ( H ) and X, ( H) , corresponding to generalized eigen-
values inside and outside the unit circle respectively, both have di-
mension n. Let us focus on the stable subspace X, ( H) . There exist
n x n matrices XI and X z such that
Then for some stable n x n matrix H, ,
Some properties of the matrix XiX2 are useful.
Lemma 1 Suppose H has no eigenvalues on 8'D. Then
(i) X: X2 is symmetric;
(ii) XiX2 2 0 i f R 2 0 andQ 2 0.
Now assume further that X I is nonsingular, i.e., the two subspaces
are complementary. Set X :=X2X; ' . Then
CH3229-2/92/0000-1111$1 .OO 0 1992 IEEE 1111
Note that the n x n matrix X is uniquely determined by the pair H
(though X1 and X2 are not), that is, H H X is a function. We shall
denote this function by Ric and write X =Ric(H).
To recap, Ric is a function R2nxZn +Rnxn that maps H to
X , where X is defined by equation (2). The domain of Ric, de-
noted domRic, consists of all symplectic pairs If with two proper-
ties, namely, H has no generalized eigenvalues on dV and the two
subspaces
- -
are complementary.
Some properties of X are given next.
Lemma 2 Suppose H E dom Ric and X =Ric(H). Then
(i) X issymmetric;
(ii) X satisfies the algebraic Riccati equation
A X(I + RX)-A - X +Q =0;
(iii) (I +RX)-A is stable.
Lemma 2 is quite standard, see, e.g., [16, 121. The following result
gives verifiable conditions under which H belongs to dom Ric.
Theorem 1 Suppose H has the form
with (A, B ) stabilizable and (C, A) having no unobservable modes on
dV. Then H E dom Ric and Ri c(H) 2 0.
Proof The proof that H has no generalized eigenvalues on the unit
circle follows from [16]. Now wewill show that the two subspaces
are complementary. Bring in matrices XI , Xz, Hi and write (1) as
two equations below (R =EB, Q =CC):
AX1 =Xl Hi +BBX2H;
- CCXI +X2 =AX2Hi.
(3)
(4)
We want to show that Xi is nonsingular, i.e., KerX1 =0. First, it is
claimed that Ker X1 is Hi-invariant. To prove this, let z E Ker XI .
Pre-multiply (3) by zlH,!XJ and post-multiply by z to get
z H~X;XI H~Z +z H~X;EB X~H;Z =0.
Note that since X;X1 2 0 (Lemma l), both terms on the left are 2 0.
Thus B XzHp =0. Now post-multiply (3) by z to get X1H;z =0,
i.e., Hi z E Ker X1. This proves the claim.
Now to prove that X1 is nonsingular, suppose on the contrary
that Ker X I #0. Then HilKer X1 has an eigenvalue, p, and a corre-
sponding eigenvector, z:
H,z = pz, ( 5 )
IpI < 1, 0 # I E KerX1.
Post-multiply (4) by z and use (5):
(PA - 1) X z~ =0.
If p =0, then Xzz =0. Otherwise, since EXzz =0 fromBXzHiz =
0 and ( 5) , wehave
Then stabilizability implies Xzz =0 as well. But if Xl z =0 and
Xzx =0, then z =0, a contradiction. This concludes the proof of
complementarity.
Now set X :=Ric(H). By Lemma 1 (R =EB,Q =CC,Xl =
QED
This theorem has various forms in the literature; for example, in
[14] similar results were given when the matrix A is nonsingular and
in [16] an indirect proof was given that X1 is nonsingular. Our proof
here is along the lines of a continuous-time proof in [lo].
I,Xz =X ) , x 2 0.
3 Discrete-Time Case
This section rederives in a state-space approach the perhaps-known
results for a discrete-time 2-optimal control problem.
We begin with the standard setup:
................... ..................
.......... .........
1 1 ; i v
............. .............
(We have used dotted lines for discrete signals and will reserve con-
tinuous lines for continuous signals.) The input w is standard white
noise - zero mean, unit covariance matrix. The problemis to design
a K that stabilizes G and minimizes the root-mean-square value of
C; it can be shown that this is equivalent to minimizing the norm on
Hz of the transfer matrix fromw to (.
3.1 State Feedback and Disturbance Feedforward
First weallow the controller to have full information. In this case, as
wewill see, the optimal controller is a constant state feedback with a
disturbance feedforward. With the exogenous input being some pulse
function, say, w =wo6d (WO is a constant vector and 61 the discrete
unit pulse), we can even think of U as unconstrained. The precise
problem is as follows:
Given the system equations
G : ((k +1) = A((k) +Elw(k) +&U( k) , W =WO6d
C(k) = Clt(k) +DllW(k)+Dl ZV( L)
with the assumptions
(i) (A, Bz) is stabilizable;
(ii) D;,D12 =I and D12 is nonsquare;
(iii) the matrix
has full rank VX E dV.
Solve the optimization problem
1112
Note that for ease of presentation we initially allow U to be in e,,,
the eztended space for ez; however, the optimal U , to be seen later,
will actually lie in e, . Assumptions (i) and (iii) are mild restrictions
and (ii) basically means that the system has more outputs to be con-
trolled than control inputs and the control weighting is nonsingular.
If DizD12 is nonsingular but not identity, we can normalize it by
defining the new U to be ( D ~ z D 1 ~ ) l / z ~ . The three assumptions to-
gether guarantee that the symplectic pair below belongs to domRic.
Define
Theorem 2 The unique optimal control is uOpt =FE t F1w. More-
over,
min IlCllz =IIJ Cwollz.
In contrast with the full-information continuous-time case where
the optimal control is a constant state feedback, the discrete-time
optimal control law involves a disturbance feedforward term, and this
is true even when Dl l =0.
3.2 Output Feedback
Now we study the Hz-optimal control problem posed at the start of
Section 3, where the measured output II, does not have full infor-
mation and therefore dynamic feedback is necessary. All discussion
pertains to the standard discrete-time setup. Let Tcw denote the
closed-loop system from w to (. We say a causal, finite-dimensional,
linear time-invariant controller Ii is admissible if it achieves internal
stability. Our goal is to find an admissible Ii to minimize Il&,llz.
Consider plants of the form
i ( X ) = [Vj CI DII
C2 DZI
The following assumptions are made:
(i) (A, Bz) is stabilizable and (Cz, A) is detectable;
(ii) D:2D12 =I with D12 nonsquare and Dzl D;, =I with Dzl
nonsquare;
(iii) the matrices
A - X Bz A - X B
[ CI DI Z] ' [ Cz D ~ I ]
have full rank VX E aV;
The first parts of assumptions (i)-(iii) were seen in Section 3.1. The
second parts of assumptions (i)-(iii) are dual to their first parts: To-
gether they guarantee that the symplectic pair J introduced below
belongs to dom Ric. I n assumption (ii) it is essential that the two
matrices D{,D,z and Dz l Di , be just nonsingular; for they can be
normalized via changing coordinates in U and $:
Theorem 3 The unique optimal controller is
Moreover,
9." l l kwl l ; =l l i cl l ; +Ilgrll;-
All the proofs are contained in [7]. The first term in the minimum
cost, Il j cl l i , is associated with optimal control with state feedback and
disturbance feedforward and the second, Ilprll;, with optimal filtering.
4 Sampled-Data Case
The formulas in the preceding section have direct application in
sampled-data control problems. We will look at the case when the
control signal is the output of a D/A device, but is otherwise uncon-
strained. Then the optimal control law is a sampled state feedback
with a suitable disturbance feedforward.
Consider the sampled-data setup
Here the continuous-time system G is described by the state equations
i ( t ) = Az ( t ) t Bi w( t ) t B z ~ ( t )
z ( t ) = Ciz(t) t Di i w( t ) t D124t ) .
The control input U is obtained through a zero-order hold HO with
sampling period h, processing a discrete signal v , the control se-
quence. Thus U and U satisfy
u( t ) =v(k), kh 5 t <( k +1)h.
The exogenous input w is assumed to be fixed and affects the system
only through the first sampling period. So w has support in [O,h).
For example, w could be the impulse w ( t ) =w06(t - t o ) , where WO is
a constant vector and 0 5 to <h.
Our sampled-data problem is
Define
the norm being on &[ O, w). We shall assume that G has zero initial
1113
state.
Now weuse the lifting technique in [ 5] to set the problemin the
lifted space. Following the notation in [ 5] , let & denote any finite-
dimensional Euclidean space (its dimension will be irrelevant) and K
denote &[O, h). The sequence space & ( 2+, K ) , or simply & ( K ) , is
defined to be
m
eZ(K) :=(11 : $k E K , ll$k11' <m}.
k=O
The norm for $k is the one on K and the normfor & ( K ) is given by
The lifting operator W, mapping &[O, 00) to l z( K) , is defined by
$ =wy $ k ( t ) =Y ( t +kh) , 0 5 t <h.
We denote the lifted signal Wy by g.
Now welift the system in the preceding figure to get
i = G [ E ]
G = WG [ Wil io].
Here the lifted system C satisfies the discrete-time equations [ 5] [since
w has support in [O, h), 3 is a pulse sequence in &(K)]
[(k -I- 1) = Ad((k) -k Bi& Bzdv(k), 3k =coad(k) ( 6)
f k = el ((k) +b l l c k +b 1 ' 2 u ( k ) , (7)
where [(k) :=z( kh) and the operators are given by
The system G can be regarded as a linear time-invariant system
in discrete time, with ik and t i r k being infinite-dimensional (functions
in K). Since the lifting operator is norm-preserving, the equivalent
discrete Hz problem is
m;tnIlEllz
subject to equations (6-7), the norm being on ! 2( K) .
This problem looks almost like the one we studied in Section 3,
the difference being that now we are treating operators instead of
matrices. So the derivation for the optimal control in Section 3 carries
over except for a few changes such as using operator adjoints, denoted
by *, instead of transposes.
In view of the assumptions in Section 3, we assume here that
(i) (Ad, BZd) is stabilizable;
(ii) the matrix &d12 is invertible;
(iii) the matrix operator
is injective VX E aV.
To write down the formulas, we need to normalize d1~ first. SO
define the matrix
Q =(b:2b12)-1'2 ( 8)
and vneW :=&-'U to get the normalized equations
((k +1 ) = Ad((k) -k 816 +BzdQvnew(k)
i k = C,((k) +b 11f i k +d ~ ~ Q%e w ( k ) .
We can now give the optimal control. Define
(9)
whole is a matrix (operator on &). The formulas for Rzcl can
be derived easily. Similarly, F, though involving operators, is
also a matrix. However, the feedforward gain F1 is an operator
mapping K to b; its action on a fixed 6 can be determined a
priori.
The optimal control can be written as
The optimal state feedback ( Fz( kh) ] is independent of the ex-
ogenous input w and can be realized by sampling z( t ) at the
same rate as the hold operator. In particular, if the rate of
the D/A device is chosen, one does not gain any advantage by
sampling z(1) faster, or even by measuring z( t ) continuously.
Assumption (i) is satisfied if ( A, Bz) is stabilizable in continuous
time and if the sampling is non-pathological in a certain sense,
see, e.g., [Ill.
4. I t is not hard to show that assumption (iii) is satisfied if
1
Ad- Bzd
is injective VX E OV, which can be checked easily since it is a
matrix expression.
5 Example
The theory of the preceding section is now applied to a simple setup
consisting of two motors controlled by one PC. The block diagram
for the system is as follows:
1114
I W
- 10 0 -10 0
c,= 0 0 0 0 0 0 " 1, 0 D l , = \ i - ,
, o o 0 0
0 0
D12=\ ! 1.
0 0.1
I I
U 2
Shown are two identical motors, with shaft angles 81 and 82. The
left-hand motor is forced by an external torque W. The controller,
A', inputs the two shaft positions and their velocities, and outputs
two voltages, u1 and u2, to the motors. The goal is that the system
should act like a telerobot: When a human applies a torque w, the
"mwter" (left-hand) motor should turn appropriately and the "slaven
(right-hand) motor should follow it.
The state vector is taken to he
I =[e, i, 82 e 2 y
For certain values of the physical parameters, the state matrices are
0 1 0 0
A = [ ! -2r1 1.
0 -24.51
0, 1 >0.1
and the result is shown in Figure 1 (0, solid, Bz dash, in degrees
versus time in seconds).
gain F one must compute the matrices
Turning to the optimal sampled-data control, for the state-feedback
Figure 1: Optimal analog controller
The disturbance-feedforward gain is an operator K +E , but since
w( t ) here is constant over [0, h) , the action of Fl is to multiply by a
matrix, denoted, say, 4. This matrix is computed as follows:
These matrices, F and F1, were computed for h =0.1 (quite large,
for illustration) and the resulting sampled-data system was simulated
with the same v ( t ) as above. The responses are shown in Figure 2:
1115
Figure 2: Optimal sampled-data controller
The response of the sampled-data system is comparable to that of the
analog system, except for the DC gains. [The same weights (i.e., C1
and D12) are used for both the analog and sampled-data controller,
but in general weights that are good for the analog controller will
not necessarily be good for the sampled-data controller, and vice
versa.] However, the analog F does not even stabilize the sampled-
data system for this large h. The point is, therefore, that when h is
given and is appreciably large, the optimal sampled-data controller is
much superior to the discretized optimal analog controller. Finally,
for interest the sampled-data system with only state feedback and
not disturbance feedforward, that is,
k = O
k 2 1,
vopt(k) ={:z(kh),
was simulated and the responses are shown in Figure 3:
Figure 3: Optimal state feedback
Not surprisingly, the response is very poor: The slave motor does not
begin to move until the start of the second sampling period, by which
time the tracking error is very large.
Acknowledgement The authors wish to thank P. P. Khargonekar
and P. A. Iglesias for helpful discussions.
References
[I] B. D. 0. Anderson and J . B. Moore, Optimal Filtering, Prentice-
Hall, Englewood Cliffs, NJ , 1979.
[2] M. Athans, The role and use of the stochastic linear-quadratic-
Gaussian problem in control system design, IEEE Tmns. Au-
tomat. Control, vol. 16, pp. 529-552, 1971.
[3] B. Bamieh and J. B. Pearson, A general framework for linear
periodic systems with application to H, sampled-data control,
IEEE Tmns. Automat. Control, vol. 37, pp. 418-435, 1992.
[4] B. Bamieh and J . B. Pearson, The Hz problemfor sampled-data
systems, Tech. Report No. 9104, Dept. of Elect. and Comp.
Engg., Rice Univ., 1991.
[ 5] B. Bamieh, J . B. Pearson, B. A. Francis, and A. Tannenbaum,
A lifting technique for linear periodic systems with applications
to sampled-data control, Systems and Control Letters, vol. 17,
pp. 79-88, 1991.
[6] T. Chen and B. A. Francis, Hz-optimal sampled-data control,
ZEEE Tmns. Automat. Control, vol. 36, No. 4, pp. 387-397,1991.
(71 T. Chen and B. A. Francis, State-space solutions to discrete-
time and sampled-data Hz control problems, Tech. Report No.
9205, Dept. of Elect. Engg., Univ. of Toronto, Revised August
1992.
[8] P. Dorato and A.H. Levis, Optimal linear regulators: the
discrete-time case, IEEE Tmns. Automat. Control, vol. 16, pp.
613-620,1971.
[9] J . C. Doyle, K. Glover, P. P. Khargonekar, and B. A. Francis,
State-space solutions to standard Hz and 3-1, problems, ZEEE
Tmns. Automat. Control, vol. 34, No. 8, pp. 831-847, 1989.
[lo] B. A. Francis, A Course in H, Control Theory, Springer-Verlag,
New York, 1987.
[ll] B. A. Francis and T. T. Georgiou, Stability theory for linear
time-invariant plants with periodic digital controllers, IEEE
TM~s . Automat. Control, vol. 33, No. 9, pp. 820-832, 1988.
[12] P. A. Iglesias and K. Glover, State space approach to discrete
time 31, control, Tech. Report, Dept. of Engg., Cambridge
Univ., 1990.
[13] P. P. Khargonekar and N. Sivashankar, HZ optimal control for
sampled-data systems, Pm. CDC, 1991.
[14] V. KuEera, The discrete Riccati equation of optimal control,
hybernetica, vol. 8, No. 5, pp. 430-447, 1972.
[15] B. P. Molinari, The stabilizing solution of the discrete algebraic
Riccati equation, IEEE h n s . Automat. Control, vol. 20, No.
3, pp. 396-399,1975.
[16] T. Pappas, A. J . Laub, and N. R. Sandell, J r., On the numerical
solution of the discrete-time algebraic Riccati equation, IEEE
Tmns. Automat. Control, vol. 25, No. 4, pp. 631-641, 1980.
[17] H. T. Toivonen, Sampled-data control of continuous-time sys-
tems with an 1-1, optimality criterion, Automatica, vol. 28, No.
1, pp. 45-54, 1992.
[18] H. K. Wimmer, Normal forms of symplectic pencils and the
discrete-time algebraic Riccati equation, Linear Algebm and its
Applications, vol. 147, pp. 411-440, 1991.
[19] Y. Yamamoto, A new approach to sampled-data control sys-
tems - a function space approach method, P m . CDC, 1990.
1116