v2009.01.01 - Convex Optimization

E.3. SYMMETRIC IDEMPOTENT MATRICES 649
E.2.1
Universal projector characteristic
Although projection is not necessarily orthogonal and R(P )̸⊥ R(I − P ) in
general, still for any projector P and any x∈ R m
Px + (I − P )x = x (1782)
must hold where R(I − P ) = N(P ) is the algebraic complement of R(P).
The algebraic complement of closed convex cone K , for example, is the
negative dual cone −K ∗ . (1900)
E.3 Symmetric idempotent matrices
When idempotent matrix P is symmetric, P is an orthogonal projector. In
other words, the direction of projection of point x∈ R m on subspace R(P )
is orthogonal to R(P ) ; id est, for P 2 =P ∈ S m and projection Px∈ R(P )
Px − x ⊥ R(P ) in R m (1783)
Perpendicularity is a necessary and sufficient condition for orthogonal
projection on a subspace. [92,4.9]
A condition equivalent to (1783) is: Norm of direction x −Px is the
infimum over all nonorthogonal projections of x on R(P ) ; [215,3.3] for
P 2 =P ∈ S m , R(P )= R(A) , matrices A,B, Z and positive integer k as
defined for (1762), and given x∈ R m
‖x − Px‖ 2 = ‖x − AA † x‖ 2 = inf
B∈R n×k ‖x − A(A † + BZ T )x‖ 2 (1784)
The infimum is attained for R(B)⊆ N(A) over any affine subset of
nonorthogonal projectors (1764) indexed by k .
Proof is straightforward: The vector 2-norm is a convex function. Setting
gradient of the norm-square to 0, applyingD.2,
(
A T ABZ T − A T (I − AA † ) ) xx T A = 0
⇔
(1785)
A T ABZ T xx T A = 0
because A T = A T AA † . Projector P =AA † is therefore unique; the
minimum-distance projector is the orthogonal projector, and vice versa.