v2009.01.01 - Convex Optimization

E.8. RANGE/ROWSPACE INTERPRETATION 673
E.8 Range/Rowspace interpretation
For idempotent matrices P 1 and P 2 of any rank, P 1 XP2
T is a projection
of R(X) on R(P 1 ) and a projection of R(X T ) on R(P 2 ) : For any given
X = UΣQ T ∈ R m×p , as in compact singular value decomposition (1450),
P 1 XP T 2 =
η∑
σ i P 1 u i qi T P2 T =
i=1
η∑
σ i P 1 u i (P 2 q i ) T (1880)
i=1
where η = ∆ min{m , p}. Recall u i ∈ R(X) and q i ∈ R(X T ) when the
corresponding singular value σ i is nonzero. (A.6.1) So P 1 projects u i on
R(P 1 ) while P 2 projects q i on R(P 2 ) ; id est, the range and rowspace of any
X are respectively projected on the ranges of P 1 and P 2 . E.14
E.9 Projection on convex set
Thus far we have discussed only projection on subspaces. Now we
generalize, considering projection on arbitrary convex sets in Euclidean space;
convex because projection is, then, unique minimum-distance and a convex
optimization problem:
For projection P C x of point x on any closed set C ⊆ R n it is obvious:
C ≡ {P C x | x∈ R n } (1881)
If C ⊆ R n is a closed convex set, then for each and every x∈ R n there exists
a unique point Px belonging to C that is closest to x in the Euclidean sense.
Like (1784), unique projection Px (or P C x) of a point x on convex set C
is that point in C closest to x ; [215,3.12]
‖x − Px‖ 2 = inf
y∈C ‖x − y‖ 2 (1882)
There exists a converse (in finite-dimensional Euclidean space):
E.14 When P 1 and P 2 are symmetric and R(P 1 )= R(u j ) and R(P 2 )= R(q j ) , then the j th
dyad term from the singular value decomposition of X is isolated by the projection. Yet
if R(P 2 )= R(q l ), l≠j ∈{1... η}, then P 1 XP 2 =0.