v2009.01.01 - Convex Optimization

614 APPENDIX D. MATRIX CALCULUS
Because gradient of the product (1651) requires total change with respect
to change in each entry of matrix X , the Xb vector must make an inner
product with each vector in the second dimension of the cubix (indicated by
dotted line segments);
⎡
∇ X (X T a)Xb = ⎢
⎣
⎤
a 1 0
0 a 1
⎥
a 2 0 ⎦
0 a 2
[ ]
b1 X 11 + b 2 X 12
∈ R 2×1×2
b 1 X 21 + b 2 X 22
(1655)
[ ]
a1 (b
= 1 X 11 + b 2 X 12 ) a 1 (b 1 X 21 + b 2 X 22 )
∈ R 2×2
a 2 (b 1 X 11 + b 2 X 12 ) a 2 (b 1 X 21 + b 2 X 22 )
= ab T X T
where the cubix appears as a complete 2 ×2×2 matrix. In like manner for
the second term ∇ X (g)f
⎡ ⎤
b 1 0
∇ X (Xb)X T b 2 0
[ ]
X11 a
a =
1 + X 21 a 2
⎢ ⎥
∈ R 2×1×2
⎣ 0 b 1
⎦ X 12 a 1 + X 22 a 2 (1656)
0 b 2
= X T ab T ∈ R 2×2
The solution
∇ X a T X 2 b = ab T X T + X T ab T (1657)
can be found from Table D.2.1 or verified using (1650).
D.1.2.1
Kronecker product
A partial remedy for venturing into hyperdimensional representations, such
as the cubix or quartix, is to first vectorize matrices as in (30). This device
gives rise to the Kronecker product of matrices ⊗ ; a.k.a, direct product
or tensor product. Although it sees reversal in the literature, [284,2.1] we
adopt the definition: for A∈ R m×n and B ∈ R p×q
⎡
⎤
B 11 A B 12 A · · · B 1q A
B ⊗ A =
∆ B 21 A B 22 A · · · B 2q A
⎢
⎥
⎣ . . . ⎦ ∈ Rpm×qn (1658)
B p1 A B p2 A · · · B pq A