v2009.01.01 - Convex Optimization
v2009.01.01 - Convex Optimization v2009.01.01 - Convex Optimization
682 APPENDIX E. PROJECTION where the convex cone has vertex-description (2.12.2.0.1), for A∈ R n×N K = {Ay | y ≽ 0} (1912) and where ‖y‖ ∞ ≤ 1 is the artificial bound. This is a convex optimization problem having no known closed-form solution, in general. It arises, for example, in the fitting of hearing aids designed around a programmable graphic equalizer (a filter bank whose only adjustable parameters are gain per band each bounded above by unity). [84] The problem is equivalent to a Schur-form semidefinite program (3.1.7.2) minimize y∈R N , t∈R subject to t [ tI x − Ay (x − Ay) T t ] ≽ 0 (1913) 0 ≼ y ≼ 1 E.9.3 nonexpansivity E.9.3.0.1 Theorem. Nonexpansivity. [149,2] [92,5.3] When C ⊂ R n is an arbitrary closed convex set, projector P projecting on C is nonexpansive in the sense: for any vectors x,y ∈ R n ‖Px − Py‖ ≤ ‖x − y‖ (1914) with equality when x −Px = y −Py . E.17 ⋄ Proof. [47] ‖x − y‖ 2 = ‖Px − Py‖ 2 + ‖(I − P )x − (I − P )y‖ 2 + 2〈x − Px, Px − Py〉 + 2〈y − Py , Py − Px〉 (1915) Nonnegativity of the last two terms follows directly from the unique minimum-distance projection theorem (E.9.1.0.2). E.17 This condition for equality corrects an error in [69] (where the norm is applied to each side of the condition given here) easily revealed by counter-example.
E.9. PROJECTION ON CONVEX SET 683 The foregoing proof reveals another flavor of nonexpansivity; for each and every x,y ∈ R n ‖Px − Py‖ 2 + ‖(I − P )x − (I − P )y‖ 2 ≤ ‖x − y‖ 2 (1916) Deutsch shows yet another: [92,5.5] E.9.4 ‖Px − Py‖ 2 ≤ 〈x − y , Px − Py〉 (1917) Easy projections Projecting any matrix H ∈ R n×n in the Euclidean/Frobenius sense orthogonally on the subspace of symmetric matrices S n in isomorphic R n2 amounts to taking the symmetric part of H ; (2.2.2.0.1) id est, (H+H T )/2 is the projection. To project any H ∈ R n×n orthogonally on the symmetric hollow subspace S n h in isomorphic Rn2 (2.2.3.0.1), we may take the symmetric part and then zero all entries along the main diagonal, or vice versa (because this is projection on the intersection of two subspaces); id est, (H + H T )/2 − δ 2 (H) . To project a matrix on the nonnegative orthant R m×n + , simply clip all negative entries to 0. Likewise, projection on the nonpositive orthant R m×n − sees all positive entries clipped to 0. Projection on other orthants is equally simple with appropriate clipping. Projecting on hyperplane, halfspace, slab:E.5.0.0.8. Projection of y ∈ R n on Euclidean ball B = {x∈ R n | ‖x − a‖ ≤ c} : for y ≠ a , P B y = (y − a) + a . c ‖y−a‖ Clipping in excess of |1| each entry of a point x∈ R n is equivalent to unique minimum-distance projection of x on a hypercube centered at the origin. (conferE.10.3.2) Projection of x∈ R n on a (rectangular) hyperbox: [53,8.1.1] C = {y ∈ R n | l ≼ y ≼ u, l ≺ u} (1918) ⎧ ⎨ l k , x k ≤ l k P(x) k=0...n = x k , l k ≤ x k ≤ u k (1919) ⎩ u k , x k ≥ u k
- Page 631 and 632: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 633 and 634: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 635 and 636: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 637 and 638: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 639 and 640: Appendix E Projection For any A∈
- Page 641 and 642: 641 U T = U † for orthonormal (in
- Page 643 and 644: E.1. IDEMPOTENT MATRICES 643 where
- Page 645 and 646: E.1. IDEMPOTENT MATRICES 645 order,
- Page 647 and 648: E.1. IDEMPOTENT MATRICES 647 When t
- Page 649 and 650: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 651 and 652: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 653 and 654: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 655 and 656: E.5. PROJECTION EXAMPLES 655 E.4.0.
- Page 657 and 658: E.5. PROJECTION EXAMPLES 657 a ∗
- Page 659 and 660: E.5. PROJECTION EXAMPLES 659 E.5.0.
- Page 661 and 662: E.6. VECTORIZATION INTERPRETATION,
- Page 663 and 664: E.6. VECTORIZATION INTERPRETATION,
- Page 665 and 666: E.6. VECTORIZATION INTERPRETATION,
- Page 667 and 668: E.6. VECTORIZATION INTERPRETATION,
- Page 669 and 670: E.7. ON VECTORIZED MATRICES OF HIGH
- Page 671 and 672: E.7. ON VECTORIZED MATRICES OF HIGH
- Page 673 and 674: E.8. RANGE/ROWSPACE INTERPRETATION
- Page 675 and 676: E.9. PROJECTION ON CONVEX SET 675 A
- Page 677 and 678: E.9. PROJECTION ON CONVEX SET 677 W
- Page 679 and 680: E.9. PROJECTION ON CONVEX SET 679 P
- Page 681: E.9. PROJECTION ON CONVEX SET 681 E
- Page 685 and 686: E.9. PROJECTION ON CONVEX SET 685
- Page 687 and 688: E.10. ALTERNATING PROJECTION 687 E.
- Page 689 and 690: E.10. ALTERNATING PROJECTION 689 b
- Page 691 and 692: E.10. ALTERNATING PROJECTION 691 a
- Page 693 and 694: E.10. ALTERNATING PROJECTION 693 (a
- Page 695 and 696: E.10. ALTERNATING PROJECTION 695 wh
- Page 697 and 698: E.10. ALTERNATING PROJECTION 697 E.
- Page 699 and 700: E.10. ALTERNATING PROJECTION 699 10
- Page 701 and 702: E.10. ALTERNATING PROJECTION 701 E.
- Page 703 and 704: E.10. ALTERNATING PROJECTION 703 E
- Page 705 and 706: Appendix F Notation and a few defin
- Page 707 and 708: 707 a.i. c.i. l.i. w.r.t affinely i
- Page 709 and 710: 709 is or ← → t → 0 + as in
- Page 711 and 712: 711 ∑ π(γ) Ξ Π ∏ ψ(Z) D D
- Page 713 and 714: 713 R m×n Euclidean vector space o
- Page 715 and 716: 715 H − H + ∂H ∂H ∂H −
- Page 717 and 718: 717 O O sort-index matrix order of
- Page 719 and 720: (x,y) angle between vectors x and y
- Page 721 and 722: Bibliography [1] Suliman Al-Homidan
- Page 723 and 724: BIBLIOGRAPHY 723 [24] Alexander I.
- Page 725 and 726: BIBLIOGRAPHY 725 [52] Stephen Boyd,
- Page 727 and 728: BIBLIOGRAPHY 727 [78] Frank Critchl
- Page 729 and 730: BIBLIOGRAPHY 729 [105] Richard L. D
- Page 731 and 732: BIBLIOGRAPHY 731 [132] Michel X. Go
682 APPENDIX E. PROJECTION<br />
where the convex cone has vertex-description (2.12.2.0.1), for A∈ R n×N<br />
K = {Ay | y ≽ 0} (1912)<br />
and where ‖y‖ ∞ ≤ 1 is the artificial bound. This is a convex optimization<br />
problem having no known closed-form solution, in general. It arises, for<br />
example, in the fitting of hearing aids designed around a programmable<br />
graphic equalizer (a filter bank whose only adjustable parameters are gain<br />
per band each bounded above by unity). [84] The problem is equivalent to a<br />
Schur-form semidefinite program (3.1.7.2)<br />
minimize<br />
y∈R N , t∈R<br />
subject to<br />
t<br />
[<br />
tI x − Ay<br />
(x − Ay) T t<br />
]<br />
≽ 0<br />
(1913)<br />
0 ≼ y ≼ 1<br />
<br />
E.9.3<br />
nonexpansivity<br />
E.9.3.0.1 Theorem. Nonexpansivity. [149,2] [92,5.3]<br />
When C ⊂ R n is an arbitrary closed convex set, projector P projecting on C<br />
is nonexpansive in the sense: for any vectors x,y ∈ R n<br />
‖Px − Py‖ ≤ ‖x − y‖ (1914)<br />
with equality when x −Px = y −Py . E.17<br />
⋄<br />
Proof. [47]<br />
‖x − y‖ 2 = ‖Px − Py‖ 2 + ‖(I − P )x − (I − P )y‖ 2<br />
+ 2〈x − Px, Px − Py〉 + 2〈y − Py , Py − Px〉<br />
(1915)<br />
Nonnegativity of the last two terms follows directly from the unique<br />
minimum-distance projection theorem (E.9.1.0.2).<br />
<br />
E.17 This condition for equality corrects an error in [69] (where the norm is applied to each<br />
side of the condition given here) easily revealed by counter-example.