v2009.01.01 - Convex Optimization

630 APPENDIX D. MATRIX CALCULUS
D.2 Tables of gradients and derivatives
[140] [57]
When proving results for symmetric matrices algebraically, it is critical
to take gradients ignoring symmetry and to then substitute symmetric
entries afterward.
a,b∈R n , x,y ∈ R k , A,B∈ R m×n , X,Y ∈ R K×L , t,µ∈R ,
i,j,k,l,K,L,m,n,M,N are integers, unless otherwise noted.
x µ means δ(δ(x) µ ) for µ∈R ; id est, entrywise vector exponentiation.
δ is the main-diagonal linear operator (1318). x 0 ∆ = 1, X 0 ∆ = I if square.
d
dx
⎡
∆ ⎢
= ⎣
d
dx 1
.
d
dx k
⎤
⎥
⎦,
→y
dg(x) ,
→y
dg 2 (x) (directional derivativesD.1), log x , e x ,
√
|x| , sgnx, x/y (Hadamard quotient), x (entrywise square root),
etcetera, are maps f : R k → R k that maintain dimension; e.g., (A.1.1)
d
dx x−1 ∆ = ∇ x 1 T δ(x) −1 1 (1741)
For A a scalar or square matrix, we have the Taylor series [68,3.6]
e A ∆ =
∞∑
k=0
1
k! Ak (1742)
Further, [287,5.4]
e A ≻ 0 ∀A ∈ S m (1743)
For all square A and integer k
det k A = detA k (1744)