Linear Quadratic Regulator (LQR) control

Consider the CTLTI system
where D T zu D zu
J =
=
Linear Quadratic Regulator (LQR) control
˙x =Ax + Buu, x(0) = x0,
z =Czx + Dzuu.
≻ Θ and the cost function
∞
z(t) T z(t) dt,
0
∞
0
x(t) T C T
z C z
Q
x + 2x(t) T C T
z D zu
S
u(t) + u(t) T D T
zuDzu u(t) dt.
R
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 1

Linear Quadratic Regulator (LQR) control
The control that minimizes the cost function J over all possible controllers (including nonlinear
controllers) is the state feedback law u = Kx where
K = −(D T
zuDzu )−1 B T
u P + DT
zuC
z ,
and P is the stabilizing solution to the Riccati equation
A T P + P A − P Bu + C T
z D T
zu DzuD −1 T
zu Bu P + DT
zuC T
z + Cz Cz The optimal control achieves the optimal performance
J ∗ ∞
=
0
z ∗ (t) T z ∗ (t) dt = x T
0 P x0.
= Θ.
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 2

Plant (CTLTI system)
State Feedback H2 Control (primal form)
˙x =Ax + Bww + Buu, x(0) = Θ,
z =Czx + Dzww + Dzuu.
Controller (state-feedback controller)
Closed Loop System
˙x = (A + BuK)
Acl
z = (Cz + DzuK)
Ccl
u =Kx.
x + Bw
Bcl
x + Dzw
w, x(0) = Θ,
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 3
Dcl
w.

State Feedback H2 Control (primal form)
Analysis Conditions (H2 norm) ∃K ∈ R m×n such that
Hcl(K, s) 2
2
if, and only if, ∃K ∈ R m×n and P ∈ S n such that
P ≻ Θ, (A T + K T B T
u )
A T cl
P + P (A + BuK)
tr[B T
w
B T cl
Bcl
Acl
< µ
+ C T
z + KT D T
zu
C T cl
P Bw ] < µ, Dzw = Θ.
Dcl
(Cz + DzuK)
C cl
≺ Θ,
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 4

State Feedback H2 Control (primal form)
Method I (congruence + change-of-variables) In primal form we need to apply the
congruence transformations to obtain P −1 ≻ Θ and
AP −1 + P −1 A T + BuKP −1 + P −1 K T B T
u +
+ P −1 C T
z + P −1 K T D T
zu CzP −1 + DzuKP −1 ≺ Θ,
which can be transformed into
X ≻ Θ, AX + XA T + BuL + L T B T
u + XC T
z + LT D T
zu (CzX + DzuL) ≺ Θ
using the change-of-variables X := P −1 , L := KP −1 . This inequality can be transformed into
an LMI by Schur complement
⎡
⎣ AX + XAT + B u L + L T B T u XCT z + LT D T zu
CzX + DzuL −I
⎤
⎦ ≺ Θ.
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 5

State Feedback H2 Control (primal form)
The “cost inequality” can be manipulated by introducing the auxiliary matrix W as
so that if tr [W ] < µ then
W ≻ B T
wP Bw,
tr B T
wP Bw < tr [W ] < µ.
Then Schur complement can be used to convert it into and LMI in X in the form
⎡ ⎤ ⎡ ⎤
⎣ W BT w
Bw P −1
⎦ =
⎣ W BT w
Bw X
⎦ ≻ Θ.
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 6

State Feedback H2 Control (primal form)
State Feedback H2 control The CTLTI system
˙x =Ax + Bww + Buu, x(0) = Θ,
z =Czx + Dzww + Dzuu.
is stabilizable by the state feedback control u = Kx such that Hwz(s)2 2 < µ if, and only if,
Dzw = Θ and ∃X ∈ Sn , L ∈ Rm×n and W ∈ Sr such that tr [W ] < µ and
⎡
⎤ ⎡ ⎤
XCT + L T D T zu
⎣ AX + XAT + BuL + LT BT u
CzX + DzuL −I
If so, a stabilizing control gain is K = LX −1 .
Remarks
• Optimal H2 control: minimize µ.
⎦ ≺ Θ,
⎣ W Bw BT w X
⎦ ≻ Θ.
• Finsler’s Lemma provide necessary and sufficient conditions in this case (why?) even when
Dzu = Θ (why?).
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 7

Equivalence between H2 and LQR control
Assume that DT zuDzu ≻ Θ and DT zuCz = Θ. Manipulate the H2 state feedback form into the
intermediate formulation
min
X≻Θ,L
tr BwX −1
Bw
s.t. AX + XA T + BuL + L T B T
u
and define the Lagrangian function
L(X, L, Λ) := tr BwX −1
Bw +
T
Λ, AX + XA + BuL + L T B T
u
where Λ ∈ S n , Λ Θ.
+ XCT z CzX + LT D T
zuDzuL Θ.
+ XCT
z C z X + LT D T
zu D zu L .
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 8

Notice that
Equivalence between H2 and LQR control
L(X, L, Λ) =f(X, Λ) + Λ, BuL + L T B T
u + LT D T
zu D zu L ,
so that
L(X, L + ɛHL, Λ) =L(X, L, Λ) + 2ɛ Λ, BuHL + L T D T
zuDzuHL 2
+ O(ɛ ),
=L(X, L, Λ) + 2ɛ Λ Bu + L T D T
zuD T
zu , HL + O(ɛ 2 ).
Hence, ∇LL = 2Λ B T u + DT zu D zu L . A necessary condition for optimality is that
∂
L L(X, L, Λ) =∇LL = 2Λ B T
u
+ DT
zu D zu L = Θ
Constraint qualification (slater condition) and perturbations on Bw can be used to guarantee that
at the optimum Λ ≻ Θ. Therefore
∂
L
L(X, L, Λ) = Θ ⇒ BT
u
+ DT
zuDzuL = Θ,
⇒ L = − D T
zuD −1 T
zu Bu .
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 9

Also by completion-of-squares
Equivalence between H2 and LQR control
Θ AX + XA T + BuL + L T B T
u
=AX + XA T + XC T
z C z
DT zuD −1 T
zu Bu X − Bu
T T
+ L D
and using the comparison theorem X ≻ ¯X where
and L = − D T zu D zu
+ XCT
z C z X + LT D T
zu D zu L,
D T
zu D zu
zu D zu
−1 B T
u +
T
DzuD −1 T
zu Bu Θ A ¯X + ¯XA T + ¯XC T
z Cz ¯X
T
− Bu DzuD −1 T
zu Bu −1 B T u .
+ L
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 10

min
X≻Θ,L
Equivalence between H2 and LQR control
Substituting L = − DT zuD −1 T
zu Bu in the original problem we obtain
tr BwX −1
Bw
s.t. AX + XA T − Bu
Notice that at the optimum P := X −1 ≻ Θ and
T
DzuD −1 T
zu Bu
−1
Θ =X AX + XA T T
− Bu DzuD −1 T
zu Bu =A T P + P A − P Bu
T
DzuD −1 T
zu Bu P + CT
z Cz .
+ XCT z CzX Θ.
+ XCT z CzX
X −1 ,
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 11

Equivalence between H2 and LQR control
The optimal state feedback H2 control for the CTLTI system
where D T zu D zu
J =
=
˙x =Ax + x0w + Buu, x(0) = Θ,
z =Czx + Dzuu.
≻ Θ is also the one that minimizes among all controllers the cost function
∞
z(t) T z(t) dt,
0
∞
0
for the CTLTI system
x(t) T C T
z C z
Q
x + 2x(t) T C T
z D zu
S
˙x =Ax + Buu, x(0) = x0,
z =Czx + Dzuu.
u(t) + u(t) T D T
zuDzu u(t) dt
R
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 12

Plant (CTLTI system)
State Feedback H∞ Control
˙x =Ax + Bww + Buu, x(0) = Θ,
z =Czx + Dzww + Dzuu.
Controller (state-feedback controller)
Closed Loop System
˙x = (A + BuK)
Acl
z = (Cz + DzuK)
Ccl
u =Kx.
x + Bw
Bcl
x + Dzw
w, x(0) = Θ,
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 13
Dcl
w.

State Feedback H∞ Control
Analysis Conditions (H∞ norm) ∃K ∈ R m×n such that
Hcl(K, s)∞ < µ
if, and only if, ∃K ∈ Rm×n and P ∈ Sn such that (BRL)
⎡
P ≻ Θ,
⎢
⎣
A T cl P + P Acl P Bcl C T cl
B T cl P −µI DT cl
Ccl Dcl −µI
⎤
⎥
⎦
≺ Θ
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 14

State Feedback H∞ Control
Method I (congruence + change-of-variables) Apply the congruence transformation
on the BRL
⎡
P
⎢
⎣
−1 Θ
Θ
I
⎤ ⎡
Θ A
⎥ ⎢
Θ⎥
⎢
⎦ ⎣
Θ Θ I
T clP + P Acl P Bcl CT B
cl
T clP −µI DT ⎤ ⎡
P
⎥ ⎢
⎥ ⎢
cl ⎦ ⎣
Ccl Dcl −µI
−1 Θ
Θ
I
⎤
Θ
⎥
Θ⎥
⎦ ≺ Θ,
Θ Θ I
⎡
⎢
⎣
AclP −1 + P −1 A T cl Bcl P −1 C T cl
⇕
B T cl −µI D T cl
CclP −1 Dcl −µI
⎤
⎥
⎦
≺ Θ.
The matrices Bcl = Bw and Dcl = Dzw are constant matrices and the products
AclP −1 = AP −1 + BuKP −1 , CclP −1 = CzP −1 + DzuKP −1 ,
can be transformed into LMI using the change-of-variables X := P −1 , L := KX.
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 15

State Feedback H∞ Control
State Feedback H∞ control The CTLTI system
˙x =Ax + Bww + Buu, x(0) = Θ,
z =Czx + Dzww + Dzuu.
is stabilizable by the state feedback control u = Kx such that Hwz(s)∞ < µ if, and only if,
∃X ∈ Sn and L ∈ Rm×n such that
⎡
⎤
X ≻ Θ,
⎢
⎣
AX + XA T + BuL + L T B T u Bw XC T + L T D T zu
If so, a stabilizing control gain is K = LX −1 .
Remarks
• Optimal H∞ control: minimize µ.
B T w −µI D T zw
CzX + DzuL Dzw −µI
⎥
⎦ ≺ Θ.
• This result can be used to provide stabilizability analysis of uncertain systems with norm
bounded uncertainty.
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 16

State Feedback H∞ Control
Finsler’s Lemma (second version) Let x ∈ R n , Q ∈ S n , B ∈ R m×n and B ∈ R p×n
such that rank (B) < n and rank (C) < n. The following statements are equivalent:
i) x T Qx < 0, ∀Bx = Θ, Cx = Θ, x = 0.
ii) B ⊥T QB ⊥ ≺ Θ and C ⊥T QC ⊥ ≺ Θ
iii) ∃ µ ∈ R : Q − µB T B ≺ Θ, and Q − µC T C ≺ Θ.
iv) ∃ X ∈ R p×m : Q + C T X B + B T X T C ≺ Θ.
Remarks
• A proof can be found in Skelton, Iwasaki and Grigoriadis, Theorem 2.3.12, p. 29.
• The reconstruction of the variable X from i), ii) and iii) is slightly more involved.
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 17

State Feedback H∞ Control
Method II (existence conditions) The analysis conditions can be rewritten as
∃K ∈ Rm×n and P ∈ Sn such that P ≻ Θ and
⎡
A
⎢
⎣
T P + P A P Bw CT B
z
T wP −µI DT Cz Dzw
⎤ ⎡ ⎤
I
⎥ ⎢ ⎥
⎥
zw⎦
+ ⎢
⎣Θ⎥
⎦
−µI Θ
Q
CT K T
B
X
T u P Θ DT
B
⎡ ⎤
P Bu
⎢ ⎥
+ ⎢
zu ⎣ Θ ⎥
⎦
Dzu
BT
K
X T
I
Θ
C
Θ ≺ Θ
In this form the equivalence between items iv) and ii) of the second form of Finsler’s Lemma can
be used to eliminate variable K.
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 18

State Feedback H∞ Control
Computing B ⊥ and C ⊥ Assuming that D T zu D zu
Bx =
Notice that
≻ Θ is nonsingular
BT u P Θ DT
x = Θ zu
⇒ B ⊥ ⎡
⎤
I
⎢
= ⎢
⎣
Θ
T −1 T −Dzu DzuDzu Bu P
Θ
⎥
I ⎥
⎦
Θ
BT u P Θ DT
zu
⎛
⎜
⎝
x1
x2
x3
⎞
⎟
⎠
= Θ ⇒ BT
u P x1 + D T
zu x3 = Θ.
T −1
Considering x3 = Dzu DzuDzu z the above condition reduces to
B T
u P x1 + z = Θ ⇒ z = −B T
u P x1 ⇒
T
x3 = −Dzu DzuD −1 T
zu Bu P x1.
Cx =
I Θ Θ x = Θ ⇒ C ⊥ =
⎡ ⎤
Θ
⎢
⎣ I
Θ
⎥
Θ⎥
⎦
Θ I
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 19

From Finsler’s Lemma, iv), ∃ K T
such that
X
⎡
A
⎢
⎣
T P + P A P Bw CT B
z
T wP −µI DT ⎤ ⎡ ⎤
I
⎥ ⎢ ⎥
⎥
zw⎦
+ ⎢
⎣Θ⎥
⎦
Cz Dzw
−µI
Θ
Q
CT State Feedback H∞ Control
K T
X
⎡
P Bu
BT u P Θ DT
B
⎢ ⎥
+ ⎢
zu ⎣ Θ ⎥
⎦
Dzu
BT
K
X T
⇕
I
Θ
C
Θ ≺ Θ,
ii) B ⊥T QB ⊥ ≺ Θ, and C ⊥T QC ⊥ ≺ Θ
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 20
⎤

State Feedback H∞ Control
The second inequality in ii) provides
⎡
Θ
Θ ≻ ⎣
Θ
I
Θ
C
⎤
Θ
⎦
I
⊥T
⎡
A
⎢
⎣
T P + P A P Bw CT B
z
T wP −µI DT Cz Dzw
Q
⎤ ⎡ ⎤
Θ Θ
⎥ ⎢ ⎥
⎥ ⎢
zw⎦
⎣ I Θ⎥
⎦
−µI Θ I
C⊥ ⎡
⎤
,
=
⎣ −µI DT zw⎦
Dzw −µI
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 21

State Feedback H∞ Control
T −1 T Defining F := Dzu DzuDzu Bu the first inequality in ii) provides
⎡
I
Θ ≻ ⎣
Θ
⎤
T
Θ −P F
⎦
I Θ
B⊥T ⎡
A
⎢
⎣
T P + P A P Bw CT B
z
T wP −µI DT Cz Dzw
Q
⎤ ⎡
I
⎥ ⎢
⎥ ⎢
zw⎦
⎣ Θ
−µI −F P
B
⎤
Θ
⎥
I ⎥
⎦
Θ
⊥
⎡
,
=
⎣ AT P + P A − P F T Cz − CT z F P − µP F T F P P Bw − P F T Dzw
BT wP − DT zwF P −µI
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 22
⎤
⎦ .

State Feedback H∞ Control
The last inequality can be converted into an LMI using the change of variable X := P −1 and the
congruence transformation
⎡
Θ ≻ ⎣ P −1 ⎡⎛
⎤
Θ ⎢⎝
⎦ ⎢
⎣
Θ I
AT P + P A − P F T Cz − CT z F P −
µP F T ⎞
⎠ P Bw − P F
F P
T Dzw
BT wP − DT ⎤
⎡
⎥ ⎣
⎦
zwF P −µI
P −1 ⎤
Θ
⎦ ,
Θ I
⎡
⎤
=
=
⎣ AP −1 + P −1AT − F T CzP −1 − P −1C T z F − µF T F Bw − F T Dzw
BT w − DT zwF −µI
⎡
⎣ AX + XAT − F T CzX − XC T z F − µF T F Bw − F T Dzw
BT w − DT zwF −µI
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 23
⎤
⎦ .
⎦ ,

State Feedback H∞ Control
State Feedback H∞ control The CTLTI system
˙x =Ax + Bww + Buu, x(0) = Θ,
z =Czx + Dzww + Dzuu.
is stabilizable by the state feedback control u = Kx such that Hwz(s)∞ < µ if, and only if,
∃X ∈ Sn such that
⎡
⎤
with F := D zu
⎣ AX + XAT − F T CzX − XC T z F − µF T F Bw − F T Dzw
BT w − DT ⎦ ≺ Θ,
zwF −µI
⎡
⎤
T −1 T DzuDzu Bu .
⎣ −µI DT zw⎦
≺ Θ, X ≻ Θ,
Dzw −µI
280B - Linear Control Design Maurício de Oliveira Control Design III – p. 24