v2010.10.26 - Convex Optimization

708 APPENDIX E. PROJECTIONE.2 I−P , Projection on algebraic complementIt follows from the diagonalizability of idempotent matrices that I − P mustalso be a projection matrix because it too is idempotent, and because it maybe expressedm∑I − P = S(I − Φ)S −1 = (1 − φ i )s i wi T (1901)where (1 − φ i ) ∈ {1, 0} are the eigenvalues of I − P (1452) whoseeigenvectors s i ,w i are identical to those of P in (1892). A consequence ofthat complementary relationship of eigenvalues is the fact, [343,2] [338,2]for subspace projector P = P 2 ∈ R m×mR(P ) = span {s i | φ i = 1 ∀i} = span {s i | (1 − φ i ) = 0 ∀i} = N(I − P )N(P ) = span {s i | φ i = 0 ∀i} = span {s i | (1 − φ i ) = 1 ∀i} = R(I − P )R(P T ) = span {w i | φ i = 1 ∀i} = span {w i | (1 − φ i ) = 0 ∀i} = N(I − P T )N(P T ) = span {w i | φ i = 0 ∀i} = span {w i | (1 − φ i ) = 1 ∀i} = R(I − P T )(1902)that is easy to see from (1892) and (1901). Idempotent I −P thereforeprojects vectors on its range: N(P ). Because all eigenvectors of a realidempotent matrix are real and independent, the algebraic complement ofR(P ) [227,3.3] is equivalent to N(P ) ; E.6 id est,R(P )⊕N(P ) = R(P T )⊕N(P T ) = R(P T )⊕N(P ) = R(P )⊕N(P T ) = R mi=1(1903)because R(P ) ⊕ R(I −P )= R m . For idempotent P ∈ R m×m , consequently,rankP + rank(I − P ) = m (1904)E.2.0.0.1 Theorem. Rank/Trace. [393,4.1 prob.9] (confer (1921))P 2 = P⇔rankP = trP and rank(I − P ) = tr(I − P )(1905)E.6 The same phenomenon occurs with symmetric (nonidempotent) matrices, for example.When summands in A ⊕ B = R m are orthogonal vector spaces, the algebraic complementis the orthogonal complement.⋄