v2009.01.01 - Convex Optimization

596 APPENDIX C. SOME ANALYTICAL OPTIMAL RESULTS
C.2 trace, singular and eigen values
For A∈ R m×n and σ(A) denoting its singular values, the nuclear norm
(Ky Fan norm) of matrix A (confer (37),A.1.1 no.10, [177, p.200]) is
∑
i
σ(A) i = tr √ A T A = sup tr(X T A) = maximize tr(X T A)
‖X‖ 2 ≤1
X∈R m×n [ I X
subject to
X T I
= 1 2 minimize
X∈S m , Y ∈S n
subject to
]
≽ 0
trX + trY
[ ] X A
A T ≽ 0
Y
(1566)
Given singular value decomposition A = SΣQ T ∈ R m×n (A.6), then
X ⋆ = SQ T ∈ R m×n is an optimal solution to the maximization
(2.3.2.0.5) while X ⋆ = SΣS T ∈ S m and Y ⋆ = QΣQ T ∈ S n is an optimal
solution to the minimization [113]. Srebro [280] asserts
∑
i
σ(A) i = 1 minimize ‖U‖ 2 2 F + ‖V U,V
‖2 F
subject to A = UV T
(1567)
= minimize ‖U‖ F ‖V ‖ F
U,V
subject to A = UV T
C.2.0.0.1 Exercise. Optimal matrix factorization.
Prove (1567). C.1
C.1 Hint: Write A = SΣQ T ∈ R m×n and
[ ] [ X A U
A T =
Y V
]
[ U T V T ] ≽ 0 (1568)
Show U ⋆ = S √ Σ∈ R m×min{m,n} and V ⋆ = Q √ Σ∈ R n×min{m,n} , hence ‖U ⋆ ‖ 2 F = ‖V ⋆ ‖ 2 F .