v2010.10.26 - Convex Optimization

E.1. IDEMPOTENT MATRICES 703where AA † b is the orthogonal projection of vector b on R(A). Least normsolution can be interpreted as orthogonal projection of the origin 0 on affinesubset A = {x |Ax=AA † b} ; (E.5.0.0.5,E.5.0.0.6)arg minimize ‖x − 0‖ 2xsubject to x ∈ A(1885)equivalently, maximization of the Euclidean ball until it kisses A ; rather,arg dist(0, A).E.1 Idempotent matricesProjection matrices are square and defined by idempotence, P 2 =P ;[331,2.6] [204,1.3] equivalent to the condition, P be diagonalizable[202,3.3 prob.3] with eigenvalues φ i ∈ {0, 1}. [393,4.1 thm.4.1]Idempotent matrices are not necessarily symmetric. The transpose ofan idempotent matrix remains idempotent; P T P T = P T . Solely exceptingP = I , all projection matrices are neither orthogonal (B.5) or invertible.[331,3.4] The collection of all projection matrices of particular dimensiondoes not form a convex set.Suppose we wish to project nonorthogonally (obliquely) on the rangeof any particular matrix A∈ R m×n . All idempotent matrices projectingnonorthogonally on R(A) may be expressed:P = A(A † + BZ T ) ∈ R m×m (1886)where R(P )= R(A) , E.2 B ∈ R n×k for k ∈{1... m} is otherwise arbitrary,and Z ∈ R m×k is any matrix whose range is in N(A T ) ; id est,A T Z = A † Z = 0 (1887)Evidently, the collection of nonorthogonal projectors projecting on R(A) isan affine subsetP k = { A(A † + BZ T ) | B ∈ R n×k} (1888)E.2 Proof. R(P )⊆ R(A) is obvious [331,3.6]. By (141) and (142),R(A † + BZ T ) = {(A † + BZ T )y | y ∈ R m }⊇ {(A † + BZ T )y | y ∈ R(A)} = R(A T )R(P ) = {A(A † + BZ T )y | y ∈ R m }⊇ {A(A † + BZ T )y | (A † + BZ T )y ∈ R(A T )} = R(A)