v2010.10.26 - Convex Optimization
v2010.10.26 - Convex Optimization v2010.10.26 - Convex Optimization
730 APPENDIX E. PROJECTIONthen, given compatible X‖X −P 1 XP 2 ‖ F = inf ‖X −A 1 (A † 1+B 1 Z TB 1 , B 2 ∈R n×k 1 )X(A †T2 +Z 2 B2 T )A T 2 ‖ F (1993)As for all subspace projectors, range of the projector is the subspace on whichprojection is made: {P 1 Y P 2 | Y ∈ R m×p }. For projectors P 1 and P 2 of anyrank, altogether, this means projection P 1 XP 2 is unique minimum-distance,orthogonalP 1 XP 2 − X ⊥ {P 1 Y P 2 | Y ∈ R m×p } in R mp (1994)and P 1 and P 2 must each be symmetric (confer (1976)) to attain the infimum.E.7.2.0.1 Proof. Minimum Frobenius norm (1993).Defining P A 1 (A † 1 + B 1 Z T 1 ) ,inf ‖X − A 1 (A † 1 + B 1 Z1 T )X(A †T2 + Z 2 B2 T )A T 2 ‖ 2 FB 1 , B 2= inf ‖X − PX(A †T2 + Z 2 B2 T )A T 2 ‖ 2 FB 1 , B 2 ()= inf tr (X T − A 2 (A † 2 + B 2 Z2 T )X T P T )(X − PX(A †T2 + Z 2 B2 T )A T 2 )B 1 , B 2 (= inf tr X T X −X T PX(A †T2 +Z 2 B2 T )A T 2 −A 2 (A †B 1 , B 22+B 2 Z2 T )X T P T X)+A 2 (A † 2+B 2 Z2 T )X T P T PX(A †T2 +Z 2 B2 T )A T 2(1995)Necessary conditions for a global minimum are ∇ B1 =0 and ∇ B2 =0. Termsnot containing B 2 in (1995) will vanish from gradient ∇ B2 ; (D.2.3)(∇ B2 tr −X T PXZ 2 B2A T T 2 −A 2 B 2 Z2X T T P T X+A 2 A † 2X T P T PXZ 2 B2A T T 2)+A 2 B 2 Z2X T T P T PXA †T2 A T 2+A 2 B 2 Z2X T T P T PXZ 2 B2A T T 2= −2A T 2X T PXZ 2 + 2A T 2A 2 A † 2X T P T PXZ 2 +)2A T 2A 2 B 2 Z2X T T P T PXZ 2= A T 2(−X T + A 2 A † 2X T P T + A 2 B 2 Z2X T T P T PXZ 2(1996)= 0 ⇔R(B 1 )⊆ N(A 1 ) and R(B 2 )⊆ N(A 2 )(or Z 2 = 0) because A T = A T AA † . Symmetry requirement (1992) is implicit.Were instead P T (A †T2 + Z 2 B2 T )A T 2 and the gradient with respect to B 1observed, then similar results are obtained. The projector is unique.Perpendicularity (1994) establishes uniqueness [109,4.9] of projectionP 1 XP 2 on a matrix subspace. The minimum-distance projector is theorthogonal projector, and vice versa.
E.7. PROJECTION ON MATRIX SUBSPACES 731E.7.2.0.2 Example. PXP redux & N(V).Suppose we define a subspace of m ×n matrices, each elemental matrixhaving columns constituting a list whose geometric center (5.5.1.0.1) is theorigin in R m :R m×nc {Y ∈ R m×n | Y 1 = 0}= {Y ∈ R m×n | N(Y ) ⊇ 1} = {Y ∈ R m×n | R(Y T ) ⊆ N(1 T )}= {XV | X ∈ R m×n } ⊂ R m×n (1997)the nonsymmetric geometric center subspace. Further suppose V ∈ S n isa projection matrix having N(V )= R(1) and R(V ) = N(1 T ). Then linearmapping T(X)=XV is the orthogonal projection of any X ∈ R m×n on R m×ncin the Euclidean (vectorization) sense because V is symmetric, N(XV )⊇1,and R(VX T )⊆ N(1 T ).Now suppose we define a subspace of symmetric n ×n matrices each ofwhose columns constitute a list having the origin in R n as geometric center,S n c {Y ∈ S n | Y 1 = 0}= {Y ∈ S n | N(Y ) ⊇ 1} = {Y ∈ S n | R(Y ) ⊆ N(1 T )}(1998)the geometric center subspace. Further suppose V ∈ S n is a projectionmatrix, the same as before. Then V XV is the orthogonal projection ofany X ∈ S n on S n c in the Euclidean sense (1994) because V is symmetric,V XV 1=0, and R(V XV )⊆ N(1 T ). Two-sided projection is necessary onlyto remain in the ambient symmetric matrix subspace. ThenS n c = {V XV | X ∈ S n } ⊂ S n (1999)has dim S n c = n(n−1)/2 in isomorphic R n(n+1)/2 . We find its orthogonalcomplement as the aggregate of all negative directions of orthogonalprojection on S n c : the translation-invariant subspace (5.5.1.1)S n⊥c {X − V XV | X ∈ S n } ⊂ S n= {u1 T + 1u T | u∈ R n }(2000)characterized by doublet u1 T + 1u T (B.2). E.13 Defining geometric centerE.13 Proof.{X − V X V | X ∈ S n } = {X − (I − 1 n 11T )X(I − 11 T 1 n ) | X ∈ Sn }= { 1 n 11T X + X11 T 1 n − 1 n 11T X11 T 1 n | X ∈ Sn }
- Page 679 and 680: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 681 and 682: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 683 and 684: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 685 and 686: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 687 and 688: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 689 and 690: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 691 and 692: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 693 and 694: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 695 and 696: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 697 and 698: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 699 and 700: Appendix EProjectionFor any A∈ R
- Page 701 and 702: 701U T = U † for orthonormal (inc
- Page 703 and 704: E.1. IDEMPOTENT MATRICES 703where A
- Page 705 and 706: E.1. IDEMPOTENT MATRICES 705order,
- Page 707 and 708: E.1. IDEMPOTENT MATRICES 707are lin
- Page 709 and 710: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 711 and 712: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 713 and 714: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 715 and 716: E.4. ALGEBRA OF PROJECTION ON AFFIN
- Page 717 and 718: E.5. PROJECTION EXAMPLES 717a ∗ 2
- Page 719 and 720: E.5. PROJECTION EXAMPLES 719where Y
- Page 721 and 722: E.5. PROJECTION EXAMPLES 721(B.4.2)
- Page 723 and 724: E.6. VECTORIZATION INTERPRETATION,
- Page 725 and 726: E.6. VECTORIZATION INTERPRETATION,
- Page 727 and 728: E.6. VECTORIZATION INTERPRETATION,
- Page 729: E.7. PROJECTION ON MATRIX SUBSPACES
- Page 733 and 734: E.8. RANGE/ROWSPACE INTERPRETATION
- Page 735 and 736: E.9. PROJECTION ON CONVEX SET 735As
- Page 737 and 738: E.9. PROJECTION ON CONVEX SET 737Wi
- Page 739 and 740: E.9. PROJECTION ON CONVEX SET 739R(
- Page 741 and 742: E.9. PROJECTION ON CONVEX SET 741E.
- Page 743 and 744: E.9. PROJECTION ON CONVEX SET 743E.
- Page 745 and 746: E.9. PROJECTION ON CONVEX SET 745Un
- Page 747 and 748: E.9. PROJECTION ON CONVEX SET 747ac
- Page 749 and 750: E.10. ALTERNATING PROJECTION 749bC
- Page 751 and 752: E.10. ALTERNATING PROJECTION 7510
- Page 753 and 754: E.10. ALTERNATING PROJECTION 753E.1
- Page 755 and 756: E.10. ALTERNATING PROJECTION 755y 2
- Page 757 and 758: E.10. ALTERNATING PROJECTION 757Def
- Page 759 and 760: E.10. ALTERNATING PROJECTION 759Dis
- Page 761 and 762: E.10. ALTERNATING PROJECTION 761mat
- Page 763 and 764: E.10. ALTERNATING PROJECTION 763K
- Page 765 and 766: E.10. ALTERNATING PROJECTION 765E.1
- Page 767 and 768: E.10. ALTERNATING PROJECTION 767E.1
- Page 769 and 770: Appendix FNotation and a few defini
- Page 771 and 772: 771A ij or A(i, j) , ij th entry of
- Page 773 and 774: 773⊞orthogonal vector sum of sets
- Page 775 and 776: 775x +vector x whose negative entri
- Page 777 and 778: 777X point list ((76) having cardin
- Page 779 and 780: 779SDPSVDSNRdBEDMS n 1S n hS n⊥hS
E.7. PROJECTION ON MATRIX SUBSPACES 731E.7.2.0.2 Example. PXP redux & N(V).Suppose we define a subspace of m ×n matrices, each elemental matrixhaving columns constituting a list whose geometric center (5.5.1.0.1) is theorigin in R m :R m×nc {Y ∈ R m×n | Y 1 = 0}= {Y ∈ R m×n | N(Y ) ⊇ 1} = {Y ∈ R m×n | R(Y T ) ⊆ N(1 T )}= {XV | X ∈ R m×n } ⊂ R m×n (1997)the nonsymmetric geometric center subspace. Further suppose V ∈ S n isa projection matrix having N(V )= R(1) and R(V ) = N(1 T ). Then linearmapping T(X)=XV is the orthogonal projection of any X ∈ R m×n on R m×ncin the Euclidean (vectorization) sense because V is symmetric, N(XV )⊇1,and R(VX T )⊆ N(1 T ).Now suppose we define a subspace of symmetric n ×n matrices each ofwhose columns constitute a list having the origin in R n as geometric center,S n c {Y ∈ S n | Y 1 = 0}= {Y ∈ S n | N(Y ) ⊇ 1} = {Y ∈ S n | R(Y ) ⊆ N(1 T )}(1998)the geometric center subspace. Further suppose V ∈ S n is a projectionmatrix, the same as before. Then V XV is the orthogonal projection ofany X ∈ S n on S n c in the Euclidean sense (1994) because V is symmetric,V XV 1=0, and R(V XV )⊆ N(1 T ). Two-sided projection is necessary onlyto remain in the ambient symmetric matrix subspace. ThenS n c = {V XV | X ∈ S n } ⊂ S n (1999)has dim S n c = n(n−1)/2 in isomorphic R n(n+1)/2 . We find its orthogonalcomplement as the aggregate of all negative directions of orthogonalprojection on S n c : the translation-invariant subspace (5.5.1.1)S n⊥c {X − V XV | X ∈ S n } ⊂ S n= {u1 T + 1u T | u∈ R n }(2000)characterized by doublet u1 T + 1u T (B.2). E.13 Defining geometric centerE.13 Proof.{X − V X V | X ∈ S n } = {X − (I − 1 n 11T )X(I − 11 T 1 n ) | X ∈ Sn }= { 1 n 11T X + X11 T 1 n − 1 n 11T X11 T 1 n | X ∈ Sn }