v2010.10.26 - Convex Optimization

748 APPENDIX E. PROJECTIONE.10 Alternating projectionAlternating projection is an iterative technique for finding a point in theintersection of a number of arbitrary closed convex sets C k , or for findingthe distance between two nonintersecting closed convex sets. Because it cansometimes be difficult or inefficient to compute the intersection or expressit analytically, one naturally asks whether it is possible to instead project(unique minimum-distance) alternately on the individual C k ; often easierand what motivates adoption of this technique. Once a cycle of alternatingprojections (an iteration) is complete, we then iterate (repeat the cycle)until convergence. If the intersection of two closed convex sets is empty, thenby convergence we mean the iterate (the result after a cycle of alternatingprojections) settles to a point of minimum distance separating the sets.While alternating projection can find the point in the nonemptyintersection closest to a given point b , it does not necessarily find theclosest point. Finding that closest point is made dependable by an elegantlysimple enhancement via correction to the alternating projection technique:this Dykstra algorithm (2086) for projection on the intersection is one ofthe most beautiful projection algorithms ever discovered. It is accuratelyinterpreted as the discovery of what alternating projection originally soughtto accomplish: unique minimum-distance projection on the nonemptyintersection of a number of arbitrary closed convex sets C k . Alternatingprojection is, in fact, a special case of the Dykstra algorithm whose discussionwe defer untilE.10.3.E.10.0.1commutative projectorsGiven two arbitrary convex sets C 1 and C 2 and their respectiveminimum-distance projection operators P 1 and P 2 , if projectors commutefor each and every x∈ R n then it is easy to show P 1 P 2 x∈ C 1 ∩ C 2 andP 2 P 1 x∈ C 1 ∩ C 2 . When projectors commute (P 1 P 2 =P 2 P 1 ), a point in theintersection can be found in a finite number of steps; while commutativity isa sufficient condition, it is not necessary (6.8.1.1.1 for example).When C 1 and C 2 are subspaces, in particular, projectors P 1 and P 2commute if and only if P 1 P 2 = P C1 ∩ C 2or iff P 2 P 1 = P C1 ∩ C 2or iff P 1 P 2 isthe orthogonal projection on a Euclidean subspace. [109, lem.9.2] Subspaceprojectors will commute, for example, when P 1 (C 2 )⊂ C 2 or P 2 (C 1 )⊂ C 1 orC 1 ⊂ C 2 or C 2 ⊂ C 1 or C 1 ⊥ C 2 . When subspace projectors commute, this