v2010.10.26 - Convex Optimization

2.3. HULLS 672.3.2.0.2 Example. Nuclear norm ball: convex hull of rank-1 matrices.From (91), in Example 2.3.2.0.1, we learn that the convex hull of normalizedsymmetric rank-1 matrices is a slice of the positive semidefinite cone.In2.9.2.7 we find the convex hull of all symmetric rank-1 matrices to bethe entire positive semidefinite cone.In the present example we abandon symmetry; instead posing, what isthe convex hull of bounded nonsymmetric rank-1 matrices:conv{uv T | ‖uv T ‖ ≤ 1, u∈ R m , v ∈ R n } = {X ∈ R m×n | ∑ iσ(X) i ≤ 1} (94)where σ(X) is a vector of singular values. (Since ‖uv T ‖= ‖u‖‖v‖ (1610),norm of each vector constituting a dyad uv T (B.1) in the hull is effectivelybounded above by 1.)Proof. (⇐) Suppose ∑ σ(X) i ≤ 1. As inA.6, define a singularvalue decomposition: X = UΣV T where U = [u 1 ... u min{m,n} ]∈ R m×min{m,n} ,V∑= [v 1 ... v min{m,n} ]∈ R n×min{m,n} , and whose sum of singular values isσ(X)i = tr Σ = κ ≤ 1. Then we may write X = ∑ σ i√ √ κuiκ κvTi which is aconvex combination of dyads each of whose norm does not exceed 1. (Srebro)(⇒) Now suppose we are given a convex combination of dyadsX = ∑ ∑α i u i viT such that αi =1, α i ≥ 0 ∀i, and ‖u i vi T ‖≤1 ∀i.Then∑by triangle inequality for singular values [203, cor.3.4.3]σ(X)i ≤ ∑ σ(α i u i vi T )= ∑ α i ‖u i vi T ‖≤ ∑ α i .Given any particular dyad uv T p in the convex hull, because its polar −uv T pand every convex combination of the two belong to that hull, then the uniqueline containing those two points ±uv T p (their affine combination (78)) mustintersect the hull’s boundary at the normalized dyads {±uv T | ‖uv T ‖=1}.Any point formed by convex combination of dyads in the hull must thereforebe expressible as a convex combination of dyads on the boundary: Figure 23.conv{uv T | ‖uv T ‖ ≤ 1, u∈ R m , v ∈ R n } ≡ conv{uv T | ‖uv T ‖ = 1, u∈ R m , v ∈ R n }(95)id est, dyads may be normalized and the hull’s boundary contains them;∂{X ∈ R m×n | ∑ iσ(X) i ≤ 1} ⊇ {uv T | ‖uv T ‖ = 1, u∈ R m , v ∈ R n } (96)Normalized dyads constitute the set of extreme points (2.6.0.0.1) of thisnuclear norm ball which is, therefore, their convex hull.