v2009.01.01 - Convex Optimization

E.10. ALTERNATING PROJECTION 687
E.10 Alternating projection
Alternating projection is an iterative technique for finding a point in the
intersection of a number of arbitrary closed convex sets C k , or for finding
the distance between two nonintersecting closed convex sets. Because it can
sometimes be difficult or inefficient to compute the intersection or express
it analytically, one naturally asks whether it is possible to instead project
(unique minimum-distance) alternately on the individual C k , often easier.
Once a cycle of alternating projections (an iteration) is complete, we then
iterate (repeat the cycle) until convergence. If the intersection of two closed
convex sets is empty, then by convergence we mean the iterate (the result
after a cycle of alternating projections) settles to a point of minimum distance
separating the sets.
While alternating projection can find the point in the nonempty
intersection closest to a given point b , it does not necessarily find it.
Dependably finding that point is solved by an elegantly simple enhancement
to the alternating projection technique: this Dykstra algorithm (1962)
for projection on the intersection is one of the most beautiful projection
algorithms ever discovered. It is accurately interpreted as the discovery
of what alternating projection originally sought to accomplish: unique
minimum-distance projection on the nonempty intersection of a number of
arbitrary closed convex sets C k . Alternating projection is, in fact, a special
case of the Dykstra algorithm whose discussion we defer untilE.10.3.
E.10.0.1
commutative projectors
Given two arbitrary convex sets C 1 and C 2 and their respective
minimum-distance projection operators P 1 and P 2 , if projectors commute
for each and every x∈ R n then it is easy to show P 1 P 2 x∈ C 1 ∩ C 2 and
P 2 P 1 x∈ C 1 ∩ C 2 . When projectors commute (P 1 P 2 =P 2 P 1 ), a point in the
intersection can be found in a finite number of steps; while commutativity is
a sufficient condition, it is not necessary (6.8.1.1.1 for example).
When C 1 and C 2 are subspaces, in particular, projectors P 1 and P 2
commute if and only if P 1 P 2 = P C1 ∩ C 2
or iff P 2 P 1 = P C1 ∩ C 2
or iff P 1 P 2 is
the orthogonal projection on a Euclidean subspace. [92, lem.9.2] Subspace
projectors will commute, for example, when P 1 (C 2 )⊂ C 2 or P 2 (C 1 )⊂ C 1 or
C 1 ⊂ C 2 or C 2 ⊂ C 1 or C 1 ⊥ C 2 . When subspace projectors commute, this
means we can find a point in the intersection of those subspaces in a finite