v2009.01.01 - Convex Optimization

Appendix D
Matrix calculus
From too much study, and from extreme passion, cometh madnesse.
−Isaac Newton [129,5]
D.1 Directional derivative, Taylor series
D.1.1
Gradients
Gradient of a differentiable real function f(x) : R K →R with respect to its
vector argument is defined
∇ 2 f(x) =
⎢
⎣
⎡
∇f(x) =
⎢
⎣
∂ 2 f(x)
∂ 2 x 1
∂f(x)
∂x 1
∂f(x)
∂x 2
.
∂f(x)
∂x K
⎤
⎥
⎦ ∈ RK (1637)
while the second-order gradient of the twice differentiable real function with
respect to its vector argument is traditionally called the Hessian ;
⎡
⎤
∂ 2 f(x)
∂x 1 ∂x 2
· · ·
∂ 2 f(x)
∂x 2 ∂x 1
.
∂ 2 f(x)
∂x K ∂x 1
∂ 2 f(x)
∂ 2 x 2
· · ·
.
. . .
∂ 2 f(x)
∂x K ∂x 2
· · ·
∂ 2 f(x)
∂x 1 ∂x K
∂ 2 f(x)
∂x 2 ∂x K
.
∂ 2 f(x)
∂ 2 x K
∈ S K (1638)
⎥
⎦
2001 Jon Dattorro. CO&EDG version 2009.01.01. All rights reserved.
Citation: Jon Dattorro, Convex Optimization & Euclidean Distance Geometry,
Meboo Publishing USA, 2005.
609