v2009.01.01 - Convex Optimization

∇ ∂g mn(X)
∂X kl
622 APPENDIX D. MATRIX CALCULUS
x = a . A vector −υ based anywhere in domf × R pointing toward the
unique bowl-bottom is specified:
[ ] x − a
υ ∝
∈ R K × R (1689)
f(x) + b
Such a vector is
since the gradient is
υ =
⎡
⎢
⎣
∇ x f(x)
→∇ xf(x)
1
df(x)
2
⎤
⎥
⎦ (1690)
∇ x f(x) = 2(x − a) (1691)
and the directional derivative in the direction of the gradient is (1711)
D.1.5
→∇ xf(x)
df(x) = ∇ x f(x) T ∇ x f(x) = 4(x − a) T (x − a) = 4(f(x) + b) (1692)
Second directional derivative
By similar argument, it so happens: the second directional derivative is
equally simple. Given g(X) : R K×L →R M×N on open domain,
⎡
∂ 2 g mn(X)
⎤
∂X kl ∂X 12
· · ·
= ∂∇g mn(X)
∂X kl
=
⎡
∇ 2 g mn (X) =
⎢
⎣
⎡
=
⎢
⎣
⎢
⎣
∇ ∂gmn(X)
∂X 11
∇ ∂gmn(X)
∂X 21
.
∇ ∂gmn(X)
∂X K1
∂∇g mn(X)
∂X 11
∂∇g mn(X)
∂X 21
.
∂∇g mn(X)
∂X K1
∂ 2 g mn(X)
∂X kl ∂X 11
∂ 2 g mn(X)
∂X kl ∂X 21
∂ 2 g mn(X)
∂X kl ∂X K1
∂ 2 g mn(X)
∂X kl ∂X 1L
∂ 2 g mn(X) ∂
∂X kl ∂X 22
· · ·
2 g mn(X)
∂X kl ∂X 2L
∈ R K×L (1693)
⎥
. . . ⎦
∂ 2 g mn(X) ∂
∂X kl ∂X K2
· · ·
2 g mn(X)
∂X kl ∂X KL
⎤
∇ ∂gmn(X)
∂X 12
· · · ∇ ∂gmn(X)
∂X 1L
∇ ∂gmn(X)
∂X 22
· · · ∇ ∂gmn(X)
∂X 2L
∈ R K×L×K×L
⎥
. . ⎦
∇ ∂gmn(X)
∂X K2
· · · ∇ ∂gmn(X)
∂X KL
(1694)
∂∇g mn(X)
⎤
∂X 12
· · ·
∂∇g mn(X)
∂X 22
.
· · ·
∂∇g mn(X)
∂X K2
· · ·
∂∇g mn(X)
∂X 1L
∂∇g mn(X)
∂X 2L
.
∂∇g mn(X)
∂X KL
⎥
⎦