v2010.10.26 - Convex Optimization

E.3. SYMMETRIC IDEMPOTENT MATRICES 711Q∈ R m×k has orthonormal columns, then Q † = Q T by the Moore-Penroseconditions. Hence, any P having an orthonormal decomposition (E.3.4)where [331,3.3] (1607)P = QQ T , Q T Q = I (1913)R(P ) = R(Q) , N(P ) = N(Q T ) (1914)is an orthogonal projector projecting on R(Q) having, for Px∈ R(Q)(confer (1898))Px − x ⊥ R(Q) in R m (1915)From (1913), orthogonal projector P is obviously positive semidefinite(A.3.1.0.6); necessarily,P T = P , P † = P , ‖P ‖ 2 = 1, P ≽ 0 (1916)and ‖Px‖ = ‖QQ T x‖ = ‖Q T x‖ because ‖Qy‖ = ‖y‖ ∀y ∈ R k . All orthogonalprojectors are therefore nonexpansive because√〈Px, x〉 = ‖Px‖ = ‖Q T x‖ ≤ ‖x‖ ∀x∈ R m (1917)the Bessel inequality, [109] [227] with equality when x∈ R(Q).From the diagonalization of idempotent matrices (1892) on page 704P = SΦS T =m∑φ i s i s T i =i=1k∑≤ mi=1s i s T i (1918)orthogonal projection of point x on R(P ) has expression like an orthogonalexpansion [109,4.10]wherePx = QQ T x =k∑s T i xs i (1919)i=1Q = S(:,1:k) = [s 1 · · · s k ] ∈ R m×k (1920)and where the s i [sic] are orthonormal eigenvectors of symmetricidempotent P . When the domain is restricted to range of P , say x=Qξ forξ ∈ R k , then x = Px = QQ T Qξ = Qξ and expansion is unique. Otherwise,any component of x in N(Q T ) will be annihilated.