v2009.01.01 - Convex Optimization

E.1. IDEMPOTENT MATRICES 643
where AA † b is the orthogonal projection of vector b on R(A). Least norm
solution can be interpreted as orthogonal projection of the origin 0 on affine
subset A = {x |Ax=AA † b} ; (E.5.0.0.5,E.5.0.0.6)
arg minimize ‖x − 0‖ 2
x
subject to x ∈ A
(1761)
equivalently, maximization of the Euclidean ball until it kisses A ; rather,
dist(0, A).
E.1 Idempotent matrices
Projection matrices are square and defined by idempotence, P 2 =P ;
[287,2.6] [178,1.3] equivalent to the condition, P be diagonalizable
[176,3.3, prob.3] with eigenvalues φ i ∈ {0, 1}. [344,4.1, thm.4.1]
Idempotent matrices are not necessarily symmetric. The transpose of an
idempotent matrix remains idempotent; P T P T = P T . Solely excepting
P = I , all projection matrices are neither orthogonal (B.5) or invertible.
[287,3.4] The collection of all projection matrices of particular dimension
does not form a convex set.
Suppose we wish to project nonorthogonally (obliquely) on the range
of any particular matrix A∈ R m×n . All idempotent matrices projecting
nonorthogonally on R(A) may be expressed:
P = A(A † + BZ T ) ∈ R m×m (1762)
where R(P )= R(A) , E.2 B ∈ R n×k for k ∈{1... m} is otherwise arbitrary,
and Z ∈ R m×k is any matrix whose range is in N(A T ) ; id est,
A T Z = A † Z = 0 (1763)
Evidently, the collection of nonorthogonal projectors projecting on R(A) is
an affine subset
P k = { A(A † + BZ T ) | B ∈ R n×k} (1764)
E.2 Proof. R(P )⊆ R(A) is obvious [287,3.6]. By (131) and (132),
R(A † + BZ T ) = {(A † + BZ T )y | y ∈ R m }
⊇ {(A † + BZ T )y | y ∈ R(A)} = R(A T )
R(P ) = {A(A † + BZ T )y | y ∈ R m }
⊇ {A(A † + BZ T )y | (A † + BZ T )y ∈ R(A T )} = R(A)