v2010.10.26 - Convex Optimization

680 APPENDIX D. MATRIX CALCULUSυ ⎡⎢⎣∇ x f(α)→∇ xf(α)1df(α)2⎤⎥⎦f(α + t y)υ T(f(α),α)✡ ✡✡✡✡✡✡✡✡✡f(x)∂HFigure 160: Strictly convex quadratic bowl in R 2 ×R ; f(x)= x T x : R 2 → Rversus x on some open disc in R 2 . Plane slice ∂H is perpendicularto function domain. Slice intersection with domain connotes bidirectionalvector y . Slope of tangent line T at point (α , f(α)) is value of ∇ x f(α) T ydirectional derivative (1834) at α in slice direction y . Negative gradient−∇ x f(x)∈ R 2 is direction of steepest descent. [375] [219,15.6] [152] Whenvector υ ∈[ R 3 entry ] υ 3 is half directional derivative in gradient direction at αυ1and when = ∇υ x f(α) , then −υ points directly toward bowl bottom.2[277,2.1,5.4.5] [34,6.3.1] which is simplest. In case of a real functiong(X) : R K×L →RIn case g(X) : R K →R→Ydg(X) = tr ( ∇g(X) T Y ) (1831)→Ydg(X) = ∇g(X) T Y (1834)Unlike gradient, directional derivative does not expand dimension;directional derivative (1809) retains the dimensions of g . The derivativewith respect to t makes the directional derivative resemble ordinary calculus→Y(D.2); e.g., when g(X) is linear, dg(X) = g(Y ). [250,7.2]