v2007.09.13 - Convex Optimization

558 APPENDIX D. MATRIX CALCULUS⎡∇g(X) T 1=⎢⎣∂g(X)∂X 11∂g(X)∂X 21.∂g(X)∂X K1∂g(X)∂X 12· · ·∂g(X)∂X 22.· · ·∂g(X)∂X K2· · ·∂g(X)∂X 1L∂g(X)∂X 2L.∂g(X)∂X KL⎤⎥⎦ ∈ RK×L×M×N (1569)When the limit for ∆t ∈ R exists, it is easy to show by substitution ofvariables in (1566)∂g mn (X) g mn (X + ∆t Y kl eY kl = limk e T l ) − g mn(X)∂X kl ∆t→0 ∆t∈ R (1570)which may be interpreted as the change in g mn at X when the change in X klis equal to Y kl , the kl th entry of any Y ∈ R K×L . Because the total changein g mn (X) due to Y is the sum of change with respect to each and everyX kl , the mn th entry of the directional derivative is the corresponding totaldifferential [160,15.8]dg mn (X)| dX→Y= ∑ k,l∂g mn (X)∂X klY kl = tr ( ∇g mn (X) T Y ) (1571)= ∑ g mn (X + ∆t Y kl elimk e T l ) − g mn(X)(1572)∆t→0 ∆tk,lg mn (X + ∆t Y ) − g mn (X)= lim(1573)∆t→0 ∆t= d dt∣ g mn (X+ t Y ) (1574)t=0where t∈ R . Assuming finite Y , equation (1573) is called the Gâteauxdifferential [29, App.A.5] [147,D.2.1] [266,5.28] whose existence is impliedby the existence of the Fréchet differential, the sum in (1571). [181,7.2] Eachmay be understood as the change in g mn at X when the change in X is equal