v2009.01.01 - Convex Optimization

D.1. DIRECTIONAL DERIVATIVE, TAYLOR SERIES 617
These foregoing formulae remain correct when gradient produces
hyperdimensional representation:
D.1.4
First directional derivative
Assume that a differentiable function g(X) : R K×L →R M×N has continuous
first- and second-order gradients ∇g and ∇ 2 g over domg which is an open
set. We seek simple expressions for the first and second directional derivatives
in direction Y ∈R K×L →Y
: respectively, dg ∈ R M×N and dg →Y
2 ∈ R M×N .
Assuming that the limit exists, we may state the partial derivative of the
mn th entry of g with respect to the kl th entry of X ;
∂g mn (X)
∂X kl
g mn (X + ∆t e
= lim
k e T l ) − g mn(X)
∆t→0 ∆t
∈ R (1674)
where e k is the k th standard basis vector in R K while e l is the l th standard
basis vector in R L . The total number of partial derivatives equals KLMN
while the gradient is defined in their terms; the mn th entry of the gradient is
⎡
∇g mn (X) =
⎢
⎣
∂g mn(X)
∂X 11
∂g mn(X)
∂X 21
.
∂g mn(X)
∂X K1
∂g mn(X)
∂X 12
· · ·
∂g mn(X)
∂X 22
· · ·
.
∂g mn(X)
∂X K2
· · ·
∂g mn(X)
∂X 1L
∂g mn(X)
∂X 2L
.
∂g mn(X)
∂X KL
⎤
∈ R K×L (1675)
⎥
⎦
while the gradient is a quartix
⎡
∇g(X) = ⎢
⎣
⎤
∇g 11 (X) ∇g 12 (X) · · · ∇g 1N (X)
∇g 21 (X) ∇g 22 (X) · · · ∇g 2N (X)
⎥
. .
. ⎦ ∈ RM×N×K×L (1676)
∇g M1 (X) ∇g M2 (X) · · · ∇g MN (X)