v2009.01.01 - Convex Optimization

624 APPENDIX D. MATRIX CALCULUS
which can be proved by substitution of variables in (1698).
second-order total differential due to any Y ∈R K×L is
The mn th
d 2 g mn (X)| dX→Y
= ∑ i,j
∑
k,l
∂ 2 g mn (X)
Y kl Y ij = tr
(∇ X tr ( ∇g mn (X) T Y ) )
T
Y
∂X kl ∂X ij
(1700)
= ∑ ∂g mn (X + ∆t Y ) − ∂g mn (X)
lim
Y ij
∆t→0 ∂X
i,j
ij ∆t
(1701)
g mn (X + 2∆t Y ) − 2g mn (X + ∆t Y ) + g mn (X)
= lim
∆t→0
∆t 2 (1702)
= d2
dt 2 ∣
∣∣∣t=0
g mn (X+ t Y ) (1703)
Hence the second directional derivative,
→Y
dg 2 (X) ∆ =
⎡
⎢
⎣
⎡
tr
(∇tr ( ∇g 11 (X) T Y ) )
T
Y =
tr
(∇tr ( ∇g 21 (X) T Y ) )
T
Y ⎢
⎣ .
tr
(∇tr ( ∇g M1 (X) T Y ) )
T
Y
d 2 g 11 (X) d 2 g 12 (X) · · · d 2 g 1N (X)
d 2 g 21 (X) d 2 g 22 (X) · · · d 2 g 2N (X)
. .
d 2 g M1 (X) d 2 g M2 (X) · · ·
.
d 2 g MN (X)
⎤
⎥
∈ R M×N
⎦
∣
∣
dX→Y
tr
(∇tr ( ∇g 12 (X) T Y ) )
T
Y · · · tr
(∇tr ( ∇g 1N (X) T Y ) ) ⎤
T
Y
tr
(∇tr ( ∇g 22 (X) T Y ) )
T
Y · · · tr
(∇tr ( ∇g 2N (X) T Y ) )
T Y .
.
tr
(∇tr ( ∇g M2 (X) T Y ) )
T
Y · · · tr
(∇tr ( ∇g MN (X) T Y ) ⎥
) ⎦
T
Y
⎡
=
⎢
⎣
∑ ∑
i,j
k,l
∑ ∑
i,j
k,l
∂ 2 g 11 (X)
∂X kl ∂X ij
Y kl Y ij
∂ 2 g 21 (X)
∂X kl ∂X ij
Y kl Y ij
.
∑ ∑
∂ 2 g M1 (X)
∂X kl ∂X ij
Y kl Y ij
i,j
k,l
∑ ∑
∂ 2 g 12 (X)
∂X kl ∂X ij
Y kl Y ij · · ·
i,j
i,j
k,l
∑ ∑
∂ 2 g 22 (X)
∂X kl ∂X ij
Y kl Y ij · · ·
k,l
.
∑ ∑
∂ 2 g M2 (X)
∂X kl ∂X ij
Y kl Y ij · · ·
i,j
k,l
∑ ∑ ⎤
∂ 2 g 1N (X)
∂X kl ∂X ij
Y kl Y ij
i,j k,l
∑ ∑
∂ 2 g 2N (X)
∂X kl ∂X ij
Y kl Y ij
i,j k,l
. ⎥
∑ ∑ ∂ 2 g MN (X) ⎦
∂X kl ∂X ij
Y kl Y ij
i,j
k,l
(1704)