v2010.10.26 - Convex Optimization

E.3. SYMMETRIC IDEMPOTENT MATRICES 709E.2.1Universal projector characteristicAlthough projection is not necessarily orthogonal and R(P )̸⊥ R(I − P ) ingeneral, still for any projector P and any x∈ R mPx + (I − P )x = x (1906)must hold where R(I − P )= N(P ) is the algebraic complement of R(P).The algebraic complement of closed convex cone K , for example, is thenegative dual cone −K ∗ . (2023)E.3 Symmetric idempotent matricesWhen idempotent matrix P is symmetric, P is an orthogonal projector. Inother words, the direction of projection of point x∈ R m on subspace R(P )is orthogonal to R(P ) ; id est, for P 2 =P ∈ S m and projection Px∈ R(P )Px − x ⊥ R(P ) in R m (1907)(confer (1898)) Perpendicularity is a necessary and sufficient condition fororthogonal projection on a subspace. [109,4.9]A condition equivalent to (1907) is: Norm of direction x −Px is theinfimum over all nonorthogonal projections of x on R(P ) ; [250,3.3] forP 2 =P ∈ S m , R(P )= R(A) , matrices A,B, Z and positive integer k asdefined for (1886), and given x∈ R m‖x − Px‖ 2 = ‖x − AA † x‖ 2 = infB∈R n×k ‖x − A(A † + BZ T )x‖ 2 = dist(x, R(P ))(1908)The infimum is attained for R(B)⊆ N(A) over any affine subset ofnonorthogonal projectors (1888) indexed by k .Proof is straightforward: The vector 2-norm is a convex function. Settinggradient of the norm-square to 0, applyingD.2,(A T ABZ T − A T (I − AA † ) ) xx T A = 0⇔(1909)A T ABZ T xx T A = 0because A T = A T AA † . Projector P =AA † is therefore unique; theminimum-distance projector is the orthogonal projector, and vice versa.