v2010.10.26 - Convex Optimization

676 APPENDIX D. MATRIX CALCULUSFor any matrices of like size, S,Y ∈ R M×N⎡⎤S(:, 1) 0 0S ◦Y = [ δ(Y (:, 1)) · · · δ(Y (:, N)) ] ⎢ 0 S(:, 2)...⎥⎣ ...... 0 ⎦ ∈ RM×N0 0 S(:, N)(1786)which converts a Hadamard product into a standard matrix product. In thespecial case that S = s and Y = y are vectors in R MD.1.3s ◦ y = δ(s)y (1787)s T ⊗ y = ys Ts ⊗ y T = sy T (1788)Chain rules for composite matrix-functionsGiven dimensionally compatible matrix-valued functions of matrix variablef(X) and g(X) [219,15.7]∇ X g ( f(X) T) = ∇ X f T ∇ f g (1789)∇ 2 X g( f(X) T) = ∇ X(∇X f T ∇ f g ) = ∇ 2 X f ∇ f g + ∇ X f T ∇ 2f g ∇ Xf (1790)D.1.3.1Two arguments∇ X g ( f(X) T , h(X) T) = ∇ X f T ∇ f g + ∇ X h T ∇ h g (1791)D.1.3.1.1 Example. Chain rule for two arguments. [42,1.1]∇ x g ( f(x) T , h(x) T) =g ( f(x) T , h(x) T) = (f(x) + h(x)) T A(f(x) + h(x)) (1792)[ ] [ ]x1εx1f(x) = , h(x) =εx 2 x 2(1793)[ 1 00 ε][ ε 0(A +A T )(f + h) +0 1](A +A T )(f + h)(1794)