v2010.10.26 - Convex Optimization
v2010.10.26 - Convex Optimization v2010.10.26 - Convex Optimization
732 APPENDIX E. PROJECTIONmappingV(X) = −V XV 1 2(996)consistently with (996), then N(V)= R(I − V) on domain S n analogouslyto vector projectors (E.2); id est,N(V) = S n⊥c (2001)a subspace of S n whose dimension is dim S n⊥c = n in isomorphic R n(n+1)/2 .Intuitively, operator V is an orthogonal projector; any argumentduplicitously in its range is a fixed point. So, this symmetric operator’snullspace must be orthogonal to its range.Now compare the subspace of symmetric matrices having all zeros in thefirst row and columnS n 1 {Y ∈ S n | Y e 1 = 0}{[ ] [ ] }0 0T 0 0T= X | X ∈ S n0 I 0 I{ [0 √ ] T [ √ ]= 2VN Z 0 2VN | Z ∈ SN}(2002)[ ] 0 0Twhere P = is an orthogonal projector. Then, similarly, PXP is0 Ithe orthogonal projection of any X ∈ S n on S n 1 in the Euclidean sense (1994),and{[ ] [ ] }0 0S n⊥T 0 0T1 X − X | X ∈ S n ⊂ S n0 I 0 I= { (2003)ue T 1 + e 1 u T | u∈ R n}Obviously, S n 1 ⊕ S n⊥1 = S n . Because {X1 | X ∈ S n } = R n ,{X − V X V | X ∈ S n } = {1ζ T + ζ1 T − 11 T (1 T ζ 1 n ) | ζ ∈Rn }= {1ζ T (I − 11 T 12n ) + (I − 12n 11T )ζ1 T | ζ ∈R n }where I − 12n 11T is invertible.
E.8. RANGE/ROWSPACE INTERPRETATION 733E.8 Range/Rowspace interpretationFor idempotent matrices P 1 and P 2 of any rank, P 1 XP2T is a projectionof R(X) on R(P 1 ) and a projection of R(X T ) on R(P 2 ) : For any given∑X = UΣQ T = η σ i u i qi T ∈ R m×p , as in compact SVD (1563),i=1P 1 XP T 2 =η∑σ i P 1 u i qi T P2 T =i=1η∑σ i P 1 u i (P 2 q i ) T (2004)i=1where η min{m , p}. Recall u i ∈ R(X) and q i ∈ R(X T ) when thecorresponding singular value σ i is nonzero. (A.6.1) So P 1 projects u i onR(P 1 ) while P 2 projects q i on R(P 2 ) ; id est, the range and rowspace of anyX are respectively projected on the ranges of P 1 and P 2 . E.14E.9 Projection on convex setThus far we have discussed only projection on subspaces. Now wegeneralize, considering projection on arbitrary convex sets in Euclidean space;convex because projection is, then, unique minimum-distance and a convexoptimization problem:For projection P C x of point x on any closed set C ⊆ R n it is obvious:C ≡ {P C x | x∈ R n } (2005)where P C is a projection operator that is convex when C is convex. [61, p.88]If C ⊆ R n is a closed convex set, then for each and every x∈ R n there existsa unique point P C x belonging to C that is closest to x in the Euclideansense. Like (1908), unique projection Px (or P C x) of a point x on convexset C is that point in C closest to x ; [250,3.12]‖x − Px‖ 2 = infy∈C ‖x − y‖ 2 = dist(x, C) (2006)There exists a converse (in finite-dimensional Euclidean space):E.14 When P 1 and P 2 are symmetric and R(P 1 )= R(u j ) and R(P 2 )= R(q j ) , then the j thdyad term from the singular value decomposition of X is isolated by the projection. Yetif R(P 2 )= R(q l ), l≠j ∈{1... η}, then P 1 XP 2 =0.
- Page 681 and 682: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 683 and 684: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 685 and 686: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 687 and 688: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 689 and 690: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 691 and 692: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 693 and 694: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 695 and 696: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 697 and 698: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 699 and 700: Appendix EProjectionFor any A∈ R
- Page 701 and 702: 701U T = U † for orthonormal (inc
- Page 703 and 704: E.1. IDEMPOTENT MATRICES 703where A
- Page 705 and 706: E.1. IDEMPOTENT MATRICES 705order,
- Page 707 and 708: E.1. IDEMPOTENT MATRICES 707are lin
- Page 709 and 710: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 711 and 712: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 713 and 714: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 715 and 716: E.4. ALGEBRA OF PROJECTION ON AFFIN
- Page 717 and 718: E.5. PROJECTION EXAMPLES 717a ∗ 2
- Page 719 and 720: E.5. PROJECTION EXAMPLES 719where Y
- Page 721 and 722: E.5. PROJECTION EXAMPLES 721(B.4.2)
- Page 723 and 724: E.6. VECTORIZATION INTERPRETATION,
- Page 725 and 726: E.6. VECTORIZATION INTERPRETATION,
- Page 727 and 728: E.6. VECTORIZATION INTERPRETATION,
- Page 729 and 730: E.7. PROJECTION ON MATRIX SUBSPACES
- Page 731: E.7. PROJECTION ON MATRIX SUBSPACES
- Page 735 and 736: E.9. PROJECTION ON CONVEX SET 735As
- Page 737 and 738: E.9. PROJECTION ON CONVEX SET 737Wi
- Page 739 and 740: E.9. PROJECTION ON CONVEX SET 739R(
- Page 741 and 742: E.9. PROJECTION ON CONVEX SET 741E.
- Page 743 and 744: E.9. PROJECTION ON CONVEX SET 743E.
- Page 745 and 746: E.9. PROJECTION ON CONVEX SET 745Un
- Page 747 and 748: E.9. PROJECTION ON CONVEX SET 747ac
- Page 749 and 750: E.10. ALTERNATING PROJECTION 749bC
- Page 751 and 752: E.10. ALTERNATING PROJECTION 7510
- Page 753 and 754: E.10. ALTERNATING PROJECTION 753E.1
- Page 755 and 756: E.10. ALTERNATING PROJECTION 755y 2
- Page 757 and 758: E.10. ALTERNATING PROJECTION 757Def
- Page 759 and 760: E.10. ALTERNATING PROJECTION 759Dis
- Page 761 and 762: E.10. ALTERNATING PROJECTION 761mat
- Page 763 and 764: E.10. ALTERNATING PROJECTION 763K
- Page 765 and 766: E.10. ALTERNATING PROJECTION 765E.1
- Page 767 and 768: E.10. ALTERNATING PROJECTION 767E.1
- Page 769 and 770: Appendix FNotation and a few defini
- Page 771 and 772: 771A ij or A(i, j) , ij th entry of
- Page 773 and 774: 773⊞orthogonal vector sum of sets
- Page 775 and 776: 775x +vector x whose negative entri
- Page 777 and 778: 777X point list ((76) having cardin
- Page 779 and 780: 779SDPSVDSNRdBEDMS n 1S n hS n⊥hS
- Page 781 and 782: 781vectorentrycubixquartixfeasible
732 APPENDIX E. PROJECTIONmappingV(X) = −V XV 1 2(996)consistently with (996), then N(V)= R(I − V) on domain S n analogouslyto vector projectors (E.2); id est,N(V) = S n⊥c (2001)a subspace of S n whose dimension is dim S n⊥c = n in isomorphic R n(n+1)/2 .Intuitively, operator V is an orthogonal projector; any argumentduplicitously in its range is a fixed point. So, this symmetric operator’snullspace must be orthogonal to its range.Now compare the subspace of symmetric matrices having all zeros in thefirst row and columnS n 1 {Y ∈ S n | Y e 1 = 0}{[ ] [ ] }0 0T 0 0T= X | X ∈ S n0 I 0 I{ [0 √ ] T [ √ ]= 2VN Z 0 2VN | Z ∈ SN}(2002)[ ] 0 0Twhere P = is an orthogonal projector. Then, similarly, PXP is0 Ithe orthogonal projection of any X ∈ S n on S n 1 in the Euclidean sense (1994),and{[ ] [ ] }0 0S n⊥T 0 0T1 X − X | X ∈ S n ⊂ S n0 I 0 I= { (2003)ue T 1 + e 1 u T | u∈ R n}Obviously, S n 1 ⊕ S n⊥1 = S n . Because {X1 | X ∈ S n } = R n ,{X − V X V | X ∈ S n } = {1ζ T + ζ1 T − 11 T (1 T ζ 1 n ) | ζ ∈Rn }= {1ζ T (I − 11 T 12n ) + (I − 12n 11T )ζ1 T | ζ ∈R n }where I − 12n 11T is invertible.