v2010.10.26 - Convex Optimization

686 APPENDIX D. MATRIX CALCULUSIn the case g(X) : R K →R has vector argument, they further simplify:and so on.D.1.7Taylor series→Ydg(X) = ∇g(X) T Y (1834)→Ydg 2 (X) = Y T ∇ 2 g(X)Y (1835)→Ydg 3 (X) = ∇ X(Y T ∇ 2 g(X)Y ) TY (1836)Series expansions of the differentiable matrix-valued function g(X) , ofmatrix argument, were given earlier in (1808) and (1829). Assuming g(X)has continuous first-, second-, and third-order gradients over the open setdomg , then for X ∈ domg and any Y ∈ R K×L the complete Taylor seriesis expressed on some open interval of µ∈Rg(X+µY ) = g(X) + µ dg(X) →Y+ 1 →Yµ2dg 2 (X) + 1 →Yµ3dg 3 (X) + o(µ 4 ) (1837)2! 3!or on some open interval of ‖Y ‖ 2g(Y ) = g(X) +→Y −X→Y −X→Y −Xdg(X) + 1 dg 2 (X) + 1 dg 3 (X) + o(‖Y ‖ 4 ) (1838)2! 3!which are third-order expansions about X . The mean value theorem fromcalculus is what insures finite order of the series. [219] [42,1.1] [41, App.A.5][199,0.4] These somewhat unbelievable formulae imply that a function canbe determined over the whole of its domain by knowing its value and all itsdirectional derivatives at a single point X .D.1.7.0.1 Example. Inverse-matrix function.Say g(Y )= Y −1 . From the table on page 692,→Ydg(X) = d dt∣ g(X+ t Y ) = −X −1 Y X −1 (1839)t=0→Ydg 2 (X) = d2dt 2 ∣∣∣∣t=0g(X+ t Y ) = 2X −1 Y X −1 Y X −1 (1840)→Ydg 3 (X) = d3dt 3 ∣∣∣∣t=0g(X+ t Y ) = −6X −1 Y X −1 Y X −1 Y X −1 (1841)