v2009.01.01 - Convex Optimization

610 APPENDIX D. MATRIX CALCULUS
The gradient of vector-valued function v(x) : R→R N on real domain is
a row-vector
[
∇v(x) =
∆
∂v 1 (x)
∂x
while the second-order gradient is
∂v 2 (x)
∂x
· · ·
∂v N (x)
∂x
]
∈ R N (1639)
[
∇ 2 v(x) =
∆
∂ 2 v 1 (x)
∂x 2
∂ 2 v 2 (x)
∂x 2 · · ·
]
∂ 2 v N (x)
∈ R N (1640)
∂x 2
Gradient of vector-valued function h(x) : R K →R N on vector domain is
⎡
∇h(x) =
∆ ⎢
⎣
∂h 1 (x)
∂x 1
∂h 1 (x)
∂x 2
.
∂h 2 (x)
∂x 1
· · ·
∂h 2 (x)
∂x 2
· · ·
.
∂h N (x)
∂x 1
∂h N (x)
∂x 2
.
⎦
(1641)
∂h 1 (x) ∂h 2 (x) ∂h
∂x K ∂x K
· · · N (x)
∂x K
= [ ∇h 1 (x) ∇h 2 (x) · · · ∇h N (x) ] ∈ R K×N
while the second-order gradient has a three-dimensional representation
dubbed cubix ; D.1
⎡
∇ 2 h(x) =
∆ ⎢
⎣
∇ ∂h 1(x)
∂x 1
∇ ∂h 1(x)
∂x 2
.
⎤
⎥
∇ ∂h 2(x)
∂x 1
· · · ∇ ∂h N(x)
∂x 1
∇ ∂h 2(x)
∂x 2
· · · ∇ ∂h N(x)
∂x 2
. .
∇ ∂h 2(x)
∂x K
· · · ∇ ∂h N(x)
⎦
(1642)
∇ ∂h 1(x)
∂x K
∂x K
= [ ∇ 2 h 1 (x) ∇ 2 h 2 (x) · · · ∇ 2 h N (x) ] ∈ R K×N×K
where the gradient of each real entry is with respect to vector x as in (1637).
⎤
⎥
D.1 The word matrix comes from the Latin for womb ; related to the prefix matri- derived
from mater meaning mother.