v2009.01.01 - Convex Optimization

D.1. DIRECTIONAL DERIVATIVE, TAYLOR SERIES 613
D.1.2
Product rules for matrix-functions
Given dimensionally compatible matrix-valued functions of matrix variable
f(X) and g(X)
while [46,8.3] [273]
∇ X
(
f(X) T g(X) ) = ∇ X (f)g + ∇ X (g)f (1649)
∇ X tr ( f(X) T g(X) ) = ∇ X
(tr ( f(X) T g(Z) ) + tr ( g(X)f(Z) T))∣ ∣
∣Z←X (1650)
These expressions implicitly apply as well to scalar-, vector-, or matrix-valued
functions of scalar, vector, or matrix arguments.
D.1.2.0.1 Example. Cubix.
Suppose f(X) : R 2×2 →R 2 = X T a and g(X) : R 2×2 →R 2 = Xb . We wish
to find
∇ X
(
f(X) T g(X) ) = ∇ X a T X 2 b (1651)
using the product rule. Formula (1649) calls for
∇ X a T X 2 b = ∇ X (X T a)Xb + ∇ X (Xb)X T a (1652)
Consider the first of the two terms:
∇ X (f)g = ∇ X (X T a)Xb
= [ ∇(X T a) 1 ∇(X T a) 2
]
Xb
(1653)
The gradient of X T a forms a cubix in R 2×2×2 .
⎡
∇ X (X T a)Xb =
⎢
⎣
∂(X T a) 1
∂X 11
∂(X T a) 1
∂X 21
∂(X T a) 2
∂X 11
∂(X T a) 1
∂(X T a) 2
∂X 12 ∂X 12
∂(X T a) 2
∂X 21
∂(X T a) 1
∂(X T a) 2
∂X 22 ∂X 22
⎤
⎡
⎢
⎣
⎥
⎦
(1654)
⎤
(Xb) 1
⎥
⎦ ∈ R 2×1×2
(Xb) 2