v2010.10.26 - Convex Optimization

Appendix DMatrix calculusFrom too much study, and from extreme passion, cometh madnesse.−Isaac Newton [154,5]D.1 Directional derivative, Taylor seriesD.1.1GradientsGradient of a differentiable real function f(x) : R K →R with respect to itsvector argument is defined uniquely in terms of partial derivatives⎡∇f(x) ⎢⎣∂f(x)∂x 1∂f(x)∂x 2.∂f(x)∂x K⎤⎥⎦ ∈ RK (1759)while the second-order gradient of the twice differentiable real function withrespect to its vector argument is traditionally called the Hessian ;⎡∇ 2 f(x) ⎢⎣∂ 2 f(x)∂x 2 1∂ 2 f(x)∂x 2 ∂x 1.∂ 2 f(x)∂x K ∂x 1∂ 2 f(x)∂x 1 ∂x 2· · ·∂ 2 f(x)∂x 2 2· · ·. . ..∂ 2 f(x)∂x K ∂x 2· · ·∂ 2 f(x)∂x 1 ∂x K∂ 2 f(x)∂x 2 ∂x K.∂ 2 f(x)∂x 2 K⎤∈ S K (1760)⎥⎦2001 Jon Dattorro. co&edg version 2010.10.26. All rights reserved.citation: Dattorro, Convex Optimization & Euclidean Distance Geometry,Mεβoo Publishing USA, 2005, v2010.10.26.669