v2009.01.01 - Convex Optimization

C.2. TRACE, SINGULAR AND EIGEN VALUES 599
C.2.0.0.2 Exercise. Rank-1 approximation.
Given symmetric matrix A∈ S N , prove:
v 1 = arg minimize ‖xx T − A‖ 2 F
x
subject to ‖x‖ = 1
(1580)
where v 1 is a normalized eigenvector of A corresponding to its largest
eigenvalue. What is the objective’s optimal value?
(Ky Fan, 1949) For B ∈ S N whose eigenvalues λ(B)∈ R N are arranged
in nonincreasing order, and for 1≤k ≤N [10,4.1] [186] [176,4.3.18]
[312,2] [206,2.1] [207]
N∑
i=N−k+1
λ(B) i = inf
U∈ R N×k
U T U=I
k∑
λ(B) i = sup
i=1
U∈ R N×k
U T U=I
tr(UU T B) = minimize
X∈ S N +
tr(XB)
subject to X ≼ I
trX = k
= maximize (k − N)µ + tr(B − Z)
µ∈R , Z∈S N +
subject to µI + Z ≽ B
tr(UU T B) = maximize
X∈ S N +
tr(XB)
subject to X ≼ I
trX = k
(a)
(b)
(c)
= minimize kµ + trZ
µ∈R , Z∈S N +
subject to µI + Z ≽ B
(d)
(1581)
Given ordered diagonalization B = QΛQ T , (A.5.2) then an optimal
U for the infimum is U ⋆ = Q(:, N − k+1:N)∈ R N×k whereas
U ⋆ = Q(:, 1:k)∈ R N×k for the supremum. In both cases, X ⋆ = U ⋆ U ⋆T .
Optimization (a) searches the convex hull of the outer product UU T
of all N ×k orthonormal matrices. (2.3.2.0.1)