Chapter 3 Geometry of convex functions - Meboo Publishing ...

3.7. GRADIENT 243
From (1749) andD.2.1, the gradient of ‖xx T − A‖ 2 F is
∇ x
(
(x T x) 2 − 2x T Ax ) = 4(x T x)x − 4Ax (568)
Setting the gradient to 0
Ax = x(x T x) (569)
is necessary for optimal solution. Replace vector x with a normalized
eigenvector v i of A∈ S N , corresponding to a positive eigenvalue λ i , scaled
by square root of that eigenvalue. Then (569) is satisfied
x ← v i
√
λi ⇒ Av i = v i λ i (570)
xx T = λ i v i vi
T is a rank-1 matrix on the positive semidefinite cone boundary,
and the minimum is achieved (7.1.2) when λ i =λ 1 is the largest positive
eigenvalue of A . If A has no positive eigenvalue, then x=0 yields the
minimum.
Differentiability is a prerequisite neither to convexity or to numerical
solution of a convex optimization problem. The gradient provides a necessary
and sufficient condition (330) (430) for optimality in the constrained case,
nevertheless, as it does in the unconstrained case:
For any differentiable multidimensional convex function, zero gradient
∇f = 0 is a necessary and sufficient condition for its unconstrained
minimization [59,5.5.3]:
3.7.0.0.2 Example. Pseudoinverse.
The pseudoinverse matrix is the unique solution to an unconstrained convex
optimization problem [155,5.5.4]: given A∈ R m×n
where
minimize
X∈R n×m ‖XA − I‖2 F (571)
‖XA − I‖ 2 F = tr ( A T X T XA − XA − A T X T + I ) (572)