v2009.01.01 - Convex Optimization

E.5. PROJECTION EXAMPLES 655
E.4.0.0.1 Theorem. Orthogonal projection on affine subset. [92,9.26]
Let A = R + α be an affine subset where α ∈ A , and let R ⊥ be the
orthogonal complement of subspace R . Then P A x is the orthogonal
projection of x∈ R n on A if and only if
P A x ∈ A , 〈P A x − x, a − α〉 = 0 ∀a ∈ A (1810)
or if and only if
P A x ∈ A , P A x − x ∈ R ⊥ (1811)
⋄
E.5 Projection examples
E.5.0.0.1 Example. Orthogonal projection on orthogonal basis.
Orthogonal projection on a subspace can instead be accomplished by
orthogonally projecting on the individual members of an orthogonal basis for
that subspace. Suppose, for example, matrix A∈ R m×n holds an orthonormal
basis for R(A) in its columns; A = ∆ [a 1 a 2 · · · a n ] . Then orthogonal
projection of vector x∈ R n on R(A) is a sum of one-dimensional orthogonal
projections
Px = AA † x = A(A T A) −1 A T x = AA T x =
n∑
a i a T i x (1812)
i=1
where each symmetric dyad a i a T i is an orthogonal projector projecting on
R(a i ). (E.6.3) Because ‖x − Px‖ is minimized by orthogonal projection,
Px is considered to be the best approximation (in the Euclidean sense) to
x from the set R(A) . [92,4.9]
E.5.0.0.2 Example. Orthogonal projection on span of nonorthogonal basis.
Orthogonal projection on a subspace can also be accomplished by projecting
nonorthogonally on the individual members of any nonorthogonal basis for
that subspace. This interpretation is in fact the principal application of the
pseudoinverse we discussed. Now suppose matrix A holds a nonorthogonal
basis for R(A) in its columns,
A = [a 1 a 2 · · · a n ] ∈ R m×n (1802)