v2009.01.01 - Convex Optimization

620 APPENDIX D. MATRIX CALCULUS
υ ∆ =
⎡
⎢
⎣
∇ x f(α)
→∇ xf(α)
1df(α)
2
⎤
⎥
⎦
f(α + t y)
υ T
(f(α),α)
✡ ✡✡✡✡✡✡✡✡✡
f(x)
∂H
Figure 136: Convex quadratic bowl in R 2 ×R ; f(x)= x T x : R 2 → R
versus x on some open disc in R 2 . Plane slice ∂H is perpendicular to
function domain. Slice intersection with domain connotes bidirectional
vector y . Slope of tangent line T at point (α , f(α)) is value of directional
derivative ∇ x f(α) T y (1711) at α in slice direction y . Negative gradient
−∇ x f(x)∈ R 2 is direction of steepest descent. [327] [190,15.6] [128] When
vector υ ∈[ R 3 entry ] υ 3 is half directional derivative in gradient direction at α
υ1
and when = ∇
υ x f(α) , then −υ points directly toward bowl bottom.
2
[239,2.1,5.4.5] [31,6.3.1] which is simplest. In case of a real function
g(X) : R K×L →R
In case g(X) : R K →R
→Y
dg(X) = tr ( ∇g(X) T Y ) (1708)
→Y
dg(X) = ∇g(X) T Y (1711)
Unlike gradient, directional derivative does not expand dimension;
directional derivative (1686) retains the dimensions of g . The derivative
with respect to t makes the directional derivative resemble ordinary calculus
→Y
(D.2); e.g., when g(X) is linear, dg(X) = g(Y ). [215,7.2]