v2009.01.01 - Convex Optimization
v2009.01.01 - Convex Optimization v2009.01.01 - Convex Optimization
688 APPENDIX E. PROJECTION b C 2 ∂K ⊥ C 1 ∩ C 2 (Pb) + Pb C 1 Figure 142: First several alternating projections (1935) in von Neumann-style projection of point b converging on closest point Pb in intersection of two closed convex sets in R 2 ; C 1 and C 2 are partially drawn in vicinity of their intersection. The pointed normal cone K ⊥ (1964) is translated to Pb , the unique minimum-distance projection of b on intersection. For this particular example, it is possible to start anywhere in a large neighborhood of b and still converge to Pb . The alternating projections are themselves robust with respect to some significant amount of noise because they belong to translated normal cone. number of steps; we find, in fact, the closest point. E.10.0.1.1 Theorem. Kronecker projector. [284,2.7] Given any projection matrices P 1 and P 2 (subspace projectors), then P 1 ⊗ P 2 and P 1 ⊗ I (1925) are projection matrices. The product preserves symmetry if present. ⋄ E.10.0.2 noncommutative projectors Typically, one considers the method of alternating projection when projectors do not commute; id est, when P 1 P 2 ≠P 2 P 1 . The iconic example for noncommutative projectors illustrated in Figure 142 shows the iterates converging to the closest point in the intersection of two arbitrary convex sets. Yet simple examples like Figure 143 reveal that noncommutative alternating projection does not
E.10. ALTERNATING PROJECTION 689 b H 1 H 2 P 2 b H 1 ∩ H 2 Pb P 1 P 2 b Figure 143: The sets {C k } in this example comprise two halfspaces H 1 and H 2 . The von Neumann-style alternating projection in R 2 quickly converges to P 1 P 2 b (feasibility). The unique minimum-distance projection on the intersection is, of course, Pb . always yield the closest point, although we shall show it always yields some point in the intersection or a point that attains the distance between two convex sets. Alternating projection is also known as successive projection [152] [149] [55], cyclic projection [123] [224,3.2], successive approximation [69], or projection on convex sets [282] [283,6.4]. It is traced back to von Neumann (1933) [322] and later Wiener [328] who showed that higher iterates of a product of two orthogonal projections on subspaces converge at each point in the ambient space to the unique minimum-distance projection on the intersection of the two subspaces. More precisely, if R 1 and R 2 are closed subspaces of a Euclidean space and P 1 and P 2 respectively denote orthogonal projection on R 1 and R 2 , then for each vector b in that space, lim (P 1P 2 ) i b = P R1 ∩ R 2 b (1926) i→∞ Deutsch [92, thm.9.8, thm.9.35] shows rate of convergence for subspaces to be geometric [342,1.4.4]; bounded above by κ 2i+1 ‖b‖ , i=0, 1, 2..., where
- Page 637 and 638: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 639 and 640: Appendix E Projection For any A∈
- Page 641 and 642: 641 U T = U † for orthonormal (in
- Page 643 and 644: E.1. IDEMPOTENT MATRICES 643 where
- Page 645 and 646: E.1. IDEMPOTENT MATRICES 645 order,
- Page 647 and 648: E.1. IDEMPOTENT MATRICES 647 When t
- Page 649 and 650: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 651 and 652: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 653 and 654: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 655 and 656: E.5. PROJECTION EXAMPLES 655 E.4.0.
- Page 657 and 658: E.5. PROJECTION EXAMPLES 657 a ∗
- Page 659 and 660: E.5. PROJECTION EXAMPLES 659 E.5.0.
- Page 661 and 662: E.6. VECTORIZATION INTERPRETATION,
- Page 663 and 664: E.6. VECTORIZATION INTERPRETATION,
- Page 665 and 666: E.6. VECTORIZATION INTERPRETATION,
- Page 667 and 668: E.6. VECTORIZATION INTERPRETATION,
- Page 669 and 670: E.7. ON VECTORIZED MATRICES OF HIGH
- Page 671 and 672: E.7. ON VECTORIZED MATRICES OF HIGH
- Page 673 and 674: E.8. RANGE/ROWSPACE INTERPRETATION
- Page 675 and 676: E.9. PROJECTION ON CONVEX SET 675 A
- Page 677 and 678: E.9. PROJECTION ON CONVEX SET 677 W
- Page 679 and 680: E.9. PROJECTION ON CONVEX SET 679 P
- Page 681 and 682: E.9. PROJECTION ON CONVEX SET 681 E
- Page 683 and 684: E.9. PROJECTION ON CONVEX SET 683 T
- Page 685 and 686: E.9. PROJECTION ON CONVEX SET 685
- Page 687: E.10. ALTERNATING PROJECTION 687 E.
- Page 691 and 692: E.10. ALTERNATING PROJECTION 691 a
- Page 693 and 694: E.10. ALTERNATING PROJECTION 693 (a
- Page 695 and 696: E.10. ALTERNATING PROJECTION 695 wh
- Page 697 and 698: E.10. ALTERNATING PROJECTION 697 E.
- Page 699 and 700: E.10. ALTERNATING PROJECTION 699 10
- Page 701 and 702: E.10. ALTERNATING PROJECTION 701 E.
- Page 703 and 704: E.10. ALTERNATING PROJECTION 703 E
- Page 705 and 706: Appendix F Notation and a few defin
- Page 707 and 708: 707 a.i. c.i. l.i. w.r.t affinely i
- Page 709 and 710: 709 is or ← → t → 0 + as in
- Page 711 and 712: 711 ∑ π(γ) Ξ Π ∏ ψ(Z) D D
- Page 713 and 714: 713 R m×n Euclidean vector space o
- Page 715 and 716: 715 H − H + ∂H ∂H ∂H −
- Page 717 and 718: 717 O O sort-index matrix order of
- Page 719 and 720: (x,y) angle between vectors x and y
- Page 721 and 722: Bibliography [1] Suliman Al-Homidan
- Page 723 and 724: BIBLIOGRAPHY 723 [24] Alexander I.
- Page 725 and 726: BIBLIOGRAPHY 725 [52] Stephen Boyd,
- Page 727 and 728: BIBLIOGRAPHY 727 [78] Frank Critchl
- Page 729 and 730: BIBLIOGRAPHY 729 [105] Richard L. D
- Page 731 and 732: BIBLIOGRAPHY 731 [132] Michel X. Go
- Page 733 and 734: BIBLIOGRAPHY 733 [162] T. Herrmann,
- Page 735 and 736: BIBLIOGRAPHY 735 [191] Mark Kahrs a
- Page 737 and 738: BIBLIOGRAPHY 737 [220] K. V. Mardia
688 APPENDIX E. PROJECTION<br />
b<br />
C 2<br />
∂K ⊥ C 1 ∩ C 2<br />
(Pb) + Pb<br />
C 1<br />
Figure 142: First several alternating projections (1935) in<br />
von Neumann-style projection of point b converging on closest point Pb<br />
in intersection of two closed convex sets in R 2 ; C 1 and C 2 are partially<br />
drawn in vicinity of their intersection. The pointed normal cone K ⊥ (1964)<br />
is translated to Pb , the unique minimum-distance projection of b on<br />
intersection. For this particular example, it is possible to start anywhere<br />
in a large neighborhood of b and still converge to Pb . The alternating<br />
projections are themselves robust with respect to some significant amount<br />
of noise because they belong to translated normal cone.<br />
number of steps; we find, in fact, the closest point.<br />
E.10.0.1.1 Theorem. Kronecker projector. [284,2.7]<br />
Given any projection matrices P 1 and P 2 (subspace projectors), then<br />
P 1 ⊗ P 2 and P 1 ⊗ I (1925)<br />
are projection matrices. The product preserves symmetry if present. ⋄<br />
E.10.0.2<br />
noncommutative projectors<br />
Typically, one considers the method of alternating projection when projectors<br />
do not commute; id est, when P 1 P 2 ≠P 2 P 1 .<br />
The iconic example for noncommutative projectors illustrated in<br />
Figure 142 shows the iterates converging to the closest point in the<br />
intersection of two arbitrary convex sets. Yet simple examples like<br />
Figure 143 reveal that noncommutative alternating projection does not