v2009.01.01 - Convex Optimization

D.1. DIRECTIONAL DERIVATIVE, TAYLOR SERIES 629
In the case of a real function g(X) : R K×L →R we have, of course,
tr
(∇ X tr ( ∇ X g(X+ t Y ) T Y ) )
T
Y = d2
dt2g(X+ t Y ) (1732)
From (1712), the simpler case, where the real function g(X) : R K →R has
vector argument,
Y T ∇ 2 X
d2
g(X+ t Y )Y =
dt2g(X+ t Y ) (1733)
D.1.8.2.1 Example. Second-order gradient.
Given real function g(X) = log detX having domain int S K + , we want to
find ∇ 2 g(X)∈ R K×K×K×K . From the tables inD.2,
h(X) ∆ = ∇g(X) = X −1 ∈ int S K + (1734)
so ∇ 2 g(X)=∇h(X). By (1721) and (1685), for Y ∈ S K
tr ( ∇h mn (X) T Y ) =
=
=
d dt∣ h mn (X+ t Y ) (1735)
( t=0
)
d dt∣
∣ h(X+ t Y ) (1736)
t=0 mn
( )
d dt∣
∣ (X+ t Y ) −1 (1737)
t=0
mn
= − ( X −1 Y X −1) mn
(1738)
Setting Y to a member of {e k e T l ∈ R K×K | k,l=1... K} , and employing a
property (32) of the trace function we find
∇ 2 g(X) mnkl = tr ( )
∇h mn (X) T e k e T l = ∇hmn (X) kl = − ( X −1 e k e T l X−1) mn
(1739)
∇ 2 g(X) kl = ∇h(X) kl = − ( X −1 e k e T l X −1) ∈ R K×K (1740)
From all these first- and second-order expressions, we may generate new
ones by evaluating both sides at arbitrary t (in some open interval) but only
after the differentiation.