v2009.01.01 - Convex Optimization

3.1. CONVEX FUNCTION 223
Setting the gradient to 0
Ax = x(x T x) (528)
is necessary for optimal solution. Replace vector x with a normalized
eigenvector v i of A∈ S N , corresponding to a positive eigenvalue λ i , scaled
by square root of that eigenvalue. Then (528) is satisfied
x ← v i
√
λi ⇒ Av i = v i λ i (529)
xx T = λ i v i vi
T is a rank-1 matrix on the positive semidefinite cone boundary,
and the minimum is achieved (7.1.2) when λ i =λ 1 is the largest positive
eigenvalue of A . If A has no positive eigenvalue, then x=0 yields the
minimum.
Differentiability is a prerequisite neither to convexity or to numerical
solution of a convex optimization problem. The gradient provides a necessary
and sufficient condition (319) (405) for optimality in the constrained case,
nevertheless, as it does in the unconstrained case:
For any differentiable multidimensional convex function, zero gradient
∇f = 0 is a necessary and sufficient condition for its unconstrained
minimization [53,5.5.3]:
3.1.8.0.2 Example. Pseudoinverse.
The pseudoinverse matrix is the unique solution to an unconstrained convex
optimization problem [134,5.5.4]: given A∈ R m×n
minimize
X∈R n×m ‖XA − I‖2 F (530)
where
‖XA − I‖ 2 F = tr ( A T X T XA − XA − A T X T + I ) (531)
whose gradient (D.2.3)
∇ X ‖XA − I‖ 2 F = 2 ( XAA T − A T) = 0 (532)
vanishes when
XAA T = A T (533)