v2010.10.26 - Convex Optimization
v2010.10.26 - Convex Optimization v2010.10.26 - Convex Optimization
716 APPENDIX E. PROJECTIONE.5.0.0.2 Example. Orthogonal projection on span of nonorthogonal basis.Orthogonal projection on a subspace can also be accomplished by projectingnonorthogonally on the individual members of any nonorthogonal basis forthat subspace. This interpretation is in fact the principal application of thepseudoinverse we discussed. Now suppose matrix A holds a nonorthogonalbasis for R(A) in its columns,A = [a 1 a 2 · · · a n ] ∈ R m×n (1927)and define the rows a ∗Ti of its pseudoinverse A † as in (1928). Thenorthogonal projection of vector x∈ R n on R(A) is a sum of one-dimensionalnonorthogonal projectionsPx = AA † x =n∑i=1a i a ∗Ti x (1938)where each nonsymmetric dyad a i a ∗Ti is a nonorthogonal projector projectingon R(a i ) , (E.6.1) idempotent because of biorthogonality condition A † A = I .The projection Px is regarded as the best approximation to x from theset R(A) , as it was in Example E.5.0.0.1.E.5.0.0.3 Example. Biorthogonal expansion as nonorthogonal projection.Biorthogonal expansion can be viewed as a sum of components, each anonorthogonal projection on the range of an extreme direction of a pointedpolyhedral cone K ; e.g., Figure 163.Suppose matrix A∈ R m×n holds a nonorthogonal basis for R(A) inits columns as in (1927), and the rows of pseudoinverse A † are definedas in (1928). Assuming the most general biorthogonality condition(A † + BZ T )A = I with BZ T defined as for (1886), then biorthogonalexpansion of vector x is a sum of one-dimensional nonorthogonal projections;for x∈ R(A)x = A(A † + BZ T )x = AA † x =n∑i=1a i a ∗Ti x (1939)where each dyad a i a ∗Ti is a nonorthogonal projector projecting on R(a i ).(E.6.1) The extreme directions of K=cone(A) are {a 1 ,..., a n } the linearlyindependent columns of A , while the extreme directions {a ∗ 1 ,..., a ∗ n}
E.5. PROJECTION EXAMPLES 717a ∗ 2K ∗a 2a ∗ 1zxK0ya 1a 1 ⊥ a ∗ 2a 2 ⊥ a ∗ 1x = y + z = P a1 x + P a2 xK ∗Figure 163: (confer Figure 63) Biorthogonal expansion of point x∈aff K isfound by projecting x nonorthogonally on extreme directions of polyhedralcone K ⊂ R 2 . (Dotted lines of projection bound this translated negated cone.)Direction of projection on extreme direction a 1 is orthogonal to extremedirection a ∗ 1 of dual cone K ∗ and parallel to a 2 (E.3.5); similarly, directionof projection on a 2 is orthogonal to a ∗ 2 and parallel to a 1 . Point x issum of nonorthogonal projections: x on R(a 1 ) and x on R(a 2 ). Expansionis unique because extreme directions of K are linearly independent. Werea 1 orthogonal to a 2 , then K would be identical to K ∗ and nonorthogonalprojections would become orthogonal.
- Page 665 and 666: C.4. TWO-SIDED ORTHOGONAL PROCRUSTE
- Page 667 and 668: C.4. TWO-SIDED ORTHOGONAL PROCRUSTE
- Page 669 and 670: Appendix DMatrix calculusFrom too m
- Page 671 and 672: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 673 and 674: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 675 and 676: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 677 and 678: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 679 and 680: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 681 and 682: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 683 and 684: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 685 and 686: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 687 and 688: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 689 and 690: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 691 and 692: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 693 and 694: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 695 and 696: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 697 and 698: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 699 and 700: Appendix EProjectionFor any A∈ R
- Page 701 and 702: 701U T = U † for orthonormal (inc
- Page 703 and 704: E.1. IDEMPOTENT MATRICES 703where A
- Page 705 and 706: E.1. IDEMPOTENT MATRICES 705order,
- Page 707 and 708: E.1. IDEMPOTENT MATRICES 707are lin
- Page 709 and 710: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 711 and 712: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 713 and 714: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 715: E.4. ALGEBRA OF PROJECTION ON AFFIN
- Page 719 and 720: E.5. PROJECTION EXAMPLES 719where Y
- Page 721 and 722: E.5. PROJECTION EXAMPLES 721(B.4.2)
- Page 723 and 724: E.6. VECTORIZATION INTERPRETATION,
- Page 725 and 726: E.6. VECTORIZATION INTERPRETATION,
- Page 727 and 728: E.6. VECTORIZATION INTERPRETATION,
- Page 729 and 730: E.7. PROJECTION ON MATRIX SUBSPACES
- Page 731 and 732: E.7. PROJECTION ON MATRIX SUBSPACES
- Page 733 and 734: E.8. RANGE/ROWSPACE INTERPRETATION
- Page 735 and 736: E.9. PROJECTION ON CONVEX SET 735As
- Page 737 and 738: E.9. PROJECTION ON CONVEX SET 737Wi
- Page 739 and 740: E.9. PROJECTION ON CONVEX SET 739R(
- Page 741 and 742: E.9. PROJECTION ON CONVEX SET 741E.
- Page 743 and 744: E.9. PROJECTION ON CONVEX SET 743E.
- Page 745 and 746: E.9. PROJECTION ON CONVEX SET 745Un
- Page 747 and 748: E.9. PROJECTION ON CONVEX SET 747ac
- Page 749 and 750: E.10. ALTERNATING PROJECTION 749bC
- Page 751 and 752: E.10. ALTERNATING PROJECTION 7510
- Page 753 and 754: E.10. ALTERNATING PROJECTION 753E.1
- Page 755 and 756: E.10. ALTERNATING PROJECTION 755y 2
- Page 757 and 758: E.10. ALTERNATING PROJECTION 757Def
- Page 759 and 760: E.10. ALTERNATING PROJECTION 759Dis
- Page 761 and 762: E.10. ALTERNATING PROJECTION 761mat
- Page 763 and 764: E.10. ALTERNATING PROJECTION 763K
- Page 765 and 766: E.10. ALTERNATING PROJECTION 765E.1
716 APPENDIX E. PROJECTIONE.5.0.0.2 Example. Orthogonal projection on span of nonorthogonal basis.Orthogonal projection on a subspace can also be accomplished by projectingnonorthogonally on the individual members of any nonorthogonal basis forthat subspace. This interpretation is in fact the principal application of thepseudoinverse we discussed. Now suppose matrix A holds a nonorthogonalbasis for R(A) in its columns,A = [a 1 a 2 · · · a n ] ∈ R m×n (1927)and define the rows a ∗Ti of its pseudoinverse A † as in (1928). Thenorthogonal projection of vector x∈ R n on R(A) is a sum of one-dimensionalnonorthogonal projectionsPx = AA † x =n∑i=1a i a ∗Ti x (1938)where each nonsymmetric dyad a i a ∗Ti is a nonorthogonal projector projectingon R(a i ) , (E.6.1) idempotent because of biorthogonality condition A † A = I .The projection Px is regarded as the best approximation to x from theset R(A) , as it was in Example E.5.0.0.1.E.5.0.0.3 Example. Biorthogonal expansion as nonorthogonal projection.Biorthogonal expansion can be viewed as a sum of components, each anonorthogonal projection on the range of an extreme direction of a pointedpolyhedral cone K ; e.g., Figure 163.Suppose matrix A∈ R m×n holds a nonorthogonal basis for R(A) inits columns as in (1927), and the rows of pseudoinverse A † are definedas in (1928). Assuming the most general biorthogonality condition(A † + BZ T )A = I with BZ T defined as for (1886), then biorthogonalexpansion of vector x is a sum of one-dimensional nonorthogonal projections;for x∈ R(A)x = A(A † + BZ T )x = AA † x =n∑i=1a i a ∗Ti x (1939)where each dyad a i a ∗Ti is a nonorthogonal projector projecting on R(a i ).(E.6.1) The extreme directions of K=cone(A) are {a 1 ,..., a n } the linearlyindependent columns of A , while the extreme directions {a ∗ 1 ,..., a ∗ n}