v2010.10.26 - Convex Optimization

E.8. RANGE/ROWSPACE INTERPRETATION 733E.8 Range/Rowspace interpretationFor idempotent matrices P 1 and P 2 of any rank, P 1 XP2T is a projectionof R(X) on R(P 1 ) and a projection of R(X T ) on R(P 2 ) : For any given∑X = UΣQ T = η σ i u i qi T ∈ R m×p , as in compact SVD (1563),i=1P 1 XP T 2 =η∑σ i P 1 u i qi T P2 T =i=1η∑σ i P 1 u i (P 2 q i ) T (2004)i=1where η min{m , p}. Recall u i ∈ R(X) and q i ∈ R(X T ) when thecorresponding singular value σ i is nonzero. (A.6.1) So P 1 projects u i onR(P 1 ) while P 2 projects q i on R(P 2 ) ; id est, the range and rowspace of anyX are respectively projected on the ranges of P 1 and P 2 . E.14E.9 Projection on convex setThus far we have discussed only projection on subspaces. Now wegeneralize, considering projection on arbitrary convex sets in Euclidean space;convex because projection is, then, unique minimum-distance and a convexoptimization problem:For projection P C x of point x on any closed set C ⊆ R n it is obvious:C ≡ {P C x | x∈ R n } (2005)where P C is a projection operator that is convex when C is convex. [61, p.88]If C ⊆ R n is a closed convex set, then for each and every x∈ R n there existsa unique point P C x belonging to C that is closest to x in the Euclideansense. Like (1908), unique projection Px (or P C x) of a point x on convexset C is that point in C closest to x ; [250,3.12]‖x − Px‖ 2 = infy∈C ‖x − y‖ 2 = dist(x, C) (2006)There exists a converse (in finite-dimensional Euclidean space):E.14 When P 1 and P 2 are symmetric and R(P 1 )= R(u j ) and R(P 2 )= R(q j ) , then the j thdyad term from the singular value decomposition of X is isolated by the projection. Yetif R(P 2 )= R(q l ), l≠j ∈{1... η}, then P 1 XP 2 =0.