v2009.01.01 - Convex Optimization

664 APPENDIX E. PROJECTION
E.6.3
Orthogonal projection on a vector
The formula for orthogonal projection of vector x on the range of vector y
(one-dimensional projection) is basic analytic geometry; [12,3.3] [287,3.2]
[318,2.2] [332,1-8]
〈y,x〉
〈y,y〉 y = yT x
y T y y = yyT
y T y x = ∆ P 1 x (1848)
where 〈y,x〉/〈y,y〉 is the coefficient of projection on R(y) . An equivalent
description is: Vector P 1 x is the orthogonal projection of vector x on
R(P 1 )= R(y). Rank-one matrix P 1 is a projection matrix because P 2 1 =P 1 .
The direction of projection is orthogonal
because P T 1 = P 1 .
E.6.4
P 1 x − x ⊥ R(P 1 ) (1849)
Orthogonal projection on a vectorized matrix
From (1848), given instead X, Y ∈ R m×n , we have the one-dimensional
orthogonal projection of matrix X in isomorphic R mn on the range of
vectorized Y : (2.2)
〈Y , X〉
〈Y , Y 〉 Y (1850)
where 〈Y , X〉/〈Y , Y 〉 is the coefficient of projection.
For orthogonal projection, the term outside the vector inner-products 〈 〉
must be identical to the terms inside in three places.
E.6.4.1
Orthogonal projection on dyad
There is opportunity for insight when Y is a dyad yz T (B.1): Instead given
X ∈ R m×n , y ∈ R m , and z ∈ R n
〈yz T , X〉
〈yz T , yz T 〉 yzT = yT Xz
y T y z T z yzT (1851)
is the one-dimensional orthogonal projection of X in isomorphic R mn on
the range of vectorized yz T . To reveal the obscured symmetric projection