v2010.10.26 - Convex Optimization

674 APPENDIX D. MATRIX CALCULUSBecause gradient of the product (1773) requires total change with respectto change in each entry of matrix X , the Xb vector must make an innerproduct with each vector in the second dimension of the cubix (indicated bydotted line segments);⎡ ⎤a 1 0∇ X (X T 0 a 1[ ]b1 Xa)Xb =11 + b 2 X 12⎢ ⎥∈ R 2×1×2⎣ a 2 0 ⎦ b 1 X 21 + b 2 X 220 a 2[ ]a1 (b= 1 X 11 + b 2 X 12 ) a 1 (b 1 X 21 + b 2 X 22 )∈ R 2×2a 2 (b 1 X 11 + b 2 X 12 ) a 2 (b 1 X 21 + b 2 X 22 )= ab T X T (1777)where the cubix appears as a complete 2 ×2×2 matrix. In like manner forthe second term ∇ X (g)f⎡ ⎤b 1 0∇ X (Xb)X T b 2 0[ ]X11 aa =1 + X 21 a 2⎢ ⎥∈ R 2×1×2⎣ 0 b 1⎦ X 12 a 1 + X 22 a 2 (1778)0 b 2= X T ab T ∈ R 2×2The solution∇ X a T X 2 b = ab T X T + X T ab T (1779)can be found from Table D.2.1 or verified using (1772).D.1.2.1Kronecker productA partial remedy for venturing into hyperdimensional matrix representations,such as the cubix or quartix, is to first vectorize matrices as in (37). Thisdevice gives rise to the Kronecker product of matrices ⊗ ; a.k.a, directproduct or tensor product. Although it sees reversal in the literature,[327,2.1] we adopt the definition: for A∈ R m×n and B ∈ R p×q⎡⎤B 11 A B 12 A · · · B 1q AB 21 A B 22 A · · · B 2q AB ⊗ A ⎢⎥⎣ . . . ⎦ ∈ Rpm×qn (1780)B p1 A B p2 A · · · B pq A