v2010.10.26 - Convex Optimization
v2010.10.26 - Convex Optimization v2010.10.26 - Convex Optimization
742 APPENDIX E. PROJECTIONwhere the inequalities denote entrywise comparison. The optimal solutionH ⋆ is simply H having all its negative entries zeroed;H ⋆ ij = max{H ij , 0} , i,j∈{1... m} × {1... n} (2032)Now suppose the nonnegative orthant is translated by T ∈ R m×n ; id est,consider R m×n+ + T . Then projection on the translated orthant is [109,4.8]H ⋆ ij = max{H ij , T ij } (2033)E.9.2.2.4 Example. Unique projection on truncated convex cone.Consider the problem of projecting a point x on a closed convex cone thatis artificially bounded; really, a bounded convex polyhedron having a vertexat the origin:minimize ‖x − Ay‖ 2y∈R Nsubject to y ≽ 0(2034)‖y‖ ∞ ≤ 1where the convex cone has vertex-description (2.12.2.0.1), for A∈ R n×NK = {Ay | y ≽ 0} (2035)and where ‖y‖ ∞ ≤ 1 is the artificial bound. This is a convex optimizationproblem having no known closed-form solution, in general. It arises, forexample, in the fitting of hearing aids designed around a programmablegraphic equalizer (a filter bank whose only adjustable parameters are gainper band each bounded above by unity). [96] The problem is equivalent to aSchur-form semidefinite program (3.5.2)minimizey∈R N , t∈Rsubject tot[tI x − Ay(x − Ay) T t]≽ 0(2036)0 ≼ y ≼ 1
E.9. PROJECTION ON CONVEX SET 743E.9.3nonexpansivityE.9.3.0.1 Theorem. Nonexpansivity. [175,2] [109,5.3]When C ⊂ R n is an arbitrary closed convex set, projector P projecting on Cis nonexpansive in the sense: for any vectors x,y ∈ R n‖Px − Py‖ ≤ ‖x − y‖ (2037)with equality when x −Px = y −Py . E.17⋄Proof. [54]‖x − y‖ 2 = ‖Px − Py‖ 2 + ‖(I − P )x − (I − P )y‖ 2+ 2〈x − Px, Px − Py〉 + 2〈y − Py , Py − Px〉(2038)Nonnegativity of the last two terms follows directly from the uniqueminimum-distance projection theorem (E.9.1.0.2).The foregoing proof reveals another flavor of nonexpansivity; for each andevery x,y ∈ R n‖Px − Py‖ 2 + ‖(I − P )x − (I − P )y‖ 2 ≤ ‖x − y‖ 2 (2039)Deutsch shows yet another: [109,5.5]E.9.4‖Px − Py‖ 2 ≤ 〈x − y , Px − Py〉 (2040)Easy projectionsTo project any matrix H ∈ R n×n orthogonally in Euclidean/Frobeniussense on subspace of symmetric matrices S n in isomorphic R n2 , takesymmetric part of H ; (2.2.2.0.1) id est, (H+H T )/2 is the projection.To project any H ∈ R n×n orthogonally on symmetric hollow subspaceS n h in isomorphic Rn2 (2.2.3.0.1,7.0.1), take symmetric part then zeroall entries along main diagonal or vice versa (because this is projectionon intersection of two subspaces); id est, (H + H T )/2 − δ 2 (H).E.17 This condition for equality corrects an error in [78] (where the norm is applied to eachside of the condition given here) easily revealed by counterexample.
- Page 691 and 692: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 693 and 694: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 695 and 696: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 697 and 698: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 699 and 700: Appendix EProjectionFor any A∈ R
- Page 701 and 702: 701U T = U † for orthonormal (inc
- Page 703 and 704: E.1. IDEMPOTENT MATRICES 703where A
- Page 705 and 706: E.1. IDEMPOTENT MATRICES 705order,
- Page 707 and 708: E.1. IDEMPOTENT MATRICES 707are lin
- Page 709 and 710: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 711 and 712: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 713 and 714: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 715 and 716: E.4. ALGEBRA OF PROJECTION ON AFFIN
- Page 717 and 718: E.5. PROJECTION EXAMPLES 717a ∗ 2
- Page 719 and 720: E.5. PROJECTION EXAMPLES 719where Y
- Page 721 and 722: E.5. PROJECTION EXAMPLES 721(B.4.2)
- Page 723 and 724: E.6. VECTORIZATION INTERPRETATION,
- Page 725 and 726: E.6. VECTORIZATION INTERPRETATION,
- Page 727 and 728: E.6. VECTORIZATION INTERPRETATION,
- Page 729 and 730: E.7. PROJECTION ON MATRIX SUBSPACES
- Page 731 and 732: E.7. PROJECTION ON MATRIX SUBSPACES
- Page 733 and 734: E.8. RANGE/ROWSPACE INTERPRETATION
- Page 735 and 736: E.9. PROJECTION ON CONVEX SET 735As
- Page 737 and 738: E.9. PROJECTION ON CONVEX SET 737Wi
- Page 739 and 740: E.9. PROJECTION ON CONVEX SET 739R(
- Page 741: E.9. PROJECTION ON CONVEX SET 741E.
- Page 745 and 746: E.9. PROJECTION ON CONVEX SET 745Un
- Page 747 and 748: E.9. PROJECTION ON CONVEX SET 747ac
- Page 749 and 750: E.10. ALTERNATING PROJECTION 749bC
- Page 751 and 752: E.10. ALTERNATING PROJECTION 7510
- Page 753 and 754: E.10. ALTERNATING PROJECTION 753E.1
- Page 755 and 756: E.10. ALTERNATING PROJECTION 755y 2
- Page 757 and 758: E.10. ALTERNATING PROJECTION 757Def
- Page 759 and 760: E.10. ALTERNATING PROJECTION 759Dis
- Page 761 and 762: E.10. ALTERNATING PROJECTION 761mat
- Page 763 and 764: E.10. ALTERNATING PROJECTION 763K
- Page 765 and 766: E.10. ALTERNATING PROJECTION 765E.1
- Page 767 and 768: E.10. ALTERNATING PROJECTION 767E.1
- Page 769 and 770: Appendix FNotation and a few defini
- Page 771 and 772: 771A ij or A(i, j) , ij th entry of
- Page 773 and 774: 773⊞orthogonal vector sum of sets
- Page 775 and 776: 775x +vector x whose negative entri
- Page 777 and 778: 777X point list ((76) having cardin
- Page 779 and 780: 779SDPSVDSNRdBEDMS n 1S n hS n⊥hS
- Page 781 and 782: 781vectorentrycubixquartixfeasible
- Page 783 and 784: 783Oorder of magnitude information
- Page 785 and 786: 785cofmatrix of cofactors correspon
- Page 787 and 788: Bibliography[1] Edwin A. Abbott. Fl
- Page 789 and 790: BIBLIOGRAPHY 789[23] Dror Baron, Mi
- Page 791 and 792: BIBLIOGRAPHY 791[49] Leonard M. Blu
742 APPENDIX E. PROJECTIONwhere the inequalities denote entrywise comparison. The optimal solutionH ⋆ is simply H having all its negative entries zeroed;H ⋆ ij = max{H ij , 0} , i,j∈{1... m} × {1... n} (2032)Now suppose the nonnegative orthant is translated by T ∈ R m×n ; id est,consider R m×n+ + T . Then projection on the translated orthant is [109,4.8]H ⋆ ij = max{H ij , T ij } (2033)E.9.2.2.4 Example. Unique projection on truncated convex cone.Consider the problem of projecting a point x on a closed convex cone thatis artificially bounded; really, a bounded convex polyhedron having a vertexat the origin:minimize ‖x − Ay‖ 2y∈R Nsubject to y ≽ 0(2034)‖y‖ ∞ ≤ 1where the convex cone has vertex-description (2.12.2.0.1), for A∈ R n×NK = {Ay | y ≽ 0} (2035)and where ‖y‖ ∞ ≤ 1 is the artificial bound. This is a convex optimizationproblem having no known closed-form solution, in general. It arises, forexample, in the fitting of hearing aids designed around a programmablegraphic equalizer (a filter bank whose only adjustable parameters are gainper band each bounded above by unity). [96] The problem is equivalent to aSchur-form semidefinite program (3.5.2)minimizey∈R N , t∈Rsubject tot[tI x − Ay(x − Ay) T t]≽ 0(2036)0 ≼ y ≼ 1