v2009.01.01 - Convex Optimization

616 APPENDIX D. MATRIX CALCULUS
which converts a Hadamard product into a standard matrix product. In the
special case that S = s and Y = y are vectors in R M
D.1.3
s ◦ y = δ(s)y (1664)
s T ⊗ y = ys T
s ⊗ y T = sy T (1665)
Chain rules for composite matrix-functions
Given dimensionally compatible matrix-valued functions of matrix variable
f(X) and g(X) [190,15.7]
∇ X g ( f(X) T) = ∇ X f T ∇ f g (1666)
∇ 2 X g( f(X) T) = ∇ X
(
∇X f T ∇ f g ) = ∇ 2 X f ∇ f g + ∇ X f T ∇ 2
f g ∇ Xf (1667)
D.1.3.1
Two arguments
∇ X g ( f(X) T , h(X) T) = ∇ X f T ∇ f g + ∇ X h T ∇ h g (1668)
D.1.3.1.1 Example. Chain rule for two arguments. [37,1.1]
∇ x g ( f(x) T , h(x) T) =
g ( f(x) T , h(x) T) = (f(x) + h(x)) T A (f(x) + h(x)) (1669)
[ ] [ ]
x1
εx1
f(x) = , h(x) =
(1670)
εx 2 x 2
∇ x g ( f(x) T , h(x) T) =
[ 1 0
0 ε
]
[ ε 0
(A +A T )(f + h) +
0 1
[ 1 + ε 0
0 1 + ε
]
(A +A T )(f + h)
] ([ ] [ ])
(A +A T x1 εx1
) +
εx 2 x 2
(1671)
(1672)
from Table D.2.1.
lim
ε→0 ∇ xg ( f(x) T , h(x) T) = (A +A T )x (1673)