v2009.01.01 - Convex Optimization
v2009.01.01 - Convex Optimization v2009.01.01 - Convex Optimization
660 APPENDIX E. PROJECTION where y p is any solution to Ay = b , and where the columns of Z ∈ R n×n−rank A constitute a basis for N(A) so that y = Zξ + y p ∈ A for all ξ ∈ R n−rank A . The infimum is found by setting the gradient of the strictly convex norm-square to 0. The minimizing argument is so and from (1786), ξ ⋆ = −(Z T Z) −1 Z T (y p − x) (1826) y ⋆ = ( I − Z(Z T Z) −1 Z T) (y p − x) + x (1827) Px = y ⋆ = x − A † (Ax − b) = (I − A † A)x + A † Ay p (1828) which is a projection of x on N(A) then translated perpendicularly with respect to the nullspace until it meets the affine subset A . E.5.0.0.7 Example. Projection on affine subset, vertex-description. Suppose now we instead describe the affine subset A in terms of some given minimal set of generators arranged columnar in X ∈ R n×N (68); id est, A ∆ = aff X = {Xa | a T 1=1} ⊆ R n (1829) Here minimal set means XV N = [x 2 −x 1 x 3 −x 1 · · · x N −x 1 ]/ √ 2 (865) is full-rank (2.4.2.2) where V N ∈ R N×N−1 is the Schoenberg auxiliary matrix (B.4.2). Then the orthogonal projection Px of any point x∈ R n on A is the solution to a minimization problem: ‖Px − x‖ 2 = inf ‖Xa − x‖ 2 a T 1=1 (1830) = inf ‖X(V N ξ + a p ) − x‖ 2 ξ∈R N−1 where a p is any solution to a T 1=1. We find the minimizing argument ξ ⋆ = −(V T NX T XV N ) −1 V T NX T (Xa p − x) (1831) and so the orthogonal projection is [179,3] Px = Xa ⋆ = (I − XV N (XV N ) † )Xa p + XV N (XV N ) † x (1832) a projection of point x on R(XV N ) then translated perpendicularly with respect to that range until it meets the affine subset A .
E.6. VECTORIZATION INTERPRETATION, 661 E.5.0.0.8 Example. Projecting on hyperplane, halfspace, slab. Given the hyperplane representation having b ∈ R and nonzero normal a∈ R m ∂H = {y | a T y = b} ⊂ R m (105) the orthogonal projection of any point x∈ R m on that hyperplane is Px = x − a(a T a) −1 (a T x − b) (1833) Orthogonal projection of x on the halfspace parametrized by b ∈ R and nonzero normal a∈ R m H − = {y | a T y ≤ b} ⊂ R m (97) is the point Px = x − a(a T a) −1 max{0, a T x − b} (1834) Orthogonal projection of x on the convex slab (Figure 11), for c < b is the point [123,5.1] B ∆ = {y | c ≤ a T y ≤ b} ⊂ R m (1835) Px = x − a(a T a) −1( max{0, a T x − b} − max{0, c − a T x} ) (1836) E.6 Vectorization interpretation, projection on a matrix E.6.1 Nonorthogonal projection on a vector Nonorthogonal projection of vector x on the range of vector y is accomplished using a normalized dyad P 0 (B.1); videlicet, 〈z,x〉 〈z,y〉 y = zT x z T y y = yzT z T y x = ∆ P 0 x (1837) where 〈z,x〉/〈z,y〉 is the coefficient of projection on y . Because P0 2 =P 0 and R(P 0 )= R(y) , rank-one matrix P 0 is a nonorthogonal projector dyad projecting on R(y). Direction of nonorthogonal projection is orthogonal to z ; id est, P 0 x − x ⊥ R(P0 T ) (1838)
- Page 609 and 610: Appendix D Matrix calculus From too
- Page 611 and 612: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 613 and 614: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 615 and 616: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 617 and 618: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 619 and 620: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 621 and 622: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 623 and 624: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 625 and 626: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 627 and 628: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 629 and 630: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 631 and 632: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 633 and 634: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 635 and 636: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 637 and 638: D.2. TABLES OF GRADIENTS AND DERIVA
- Page 639 and 640: Appendix E Projection For any A∈
- Page 641 and 642: 641 U T = U † for orthonormal (in
- Page 643 and 644: E.1. IDEMPOTENT MATRICES 643 where
- Page 645 and 646: E.1. IDEMPOTENT MATRICES 645 order,
- Page 647 and 648: E.1. IDEMPOTENT MATRICES 647 When t
- Page 649 and 650: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 651 and 652: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 653 and 654: E.3. SYMMETRIC IDEMPOTENT MATRICES
- Page 655 and 656: E.5. PROJECTION EXAMPLES 655 E.4.0.
- Page 657 and 658: E.5. PROJECTION EXAMPLES 657 a ∗
- Page 659: E.5. PROJECTION EXAMPLES 659 E.5.0.
- Page 663 and 664: E.6. VECTORIZATION INTERPRETATION,
- Page 665 and 666: E.6. VECTORIZATION INTERPRETATION,
- Page 667 and 668: E.6. VECTORIZATION INTERPRETATION,
- Page 669 and 670: E.7. ON VECTORIZED MATRICES OF HIGH
- Page 671 and 672: E.7. ON VECTORIZED MATRICES OF HIGH
- Page 673 and 674: E.8. RANGE/ROWSPACE INTERPRETATION
- Page 675 and 676: E.9. PROJECTION ON CONVEX SET 675 A
- Page 677 and 678: E.9. PROJECTION ON CONVEX SET 677 W
- Page 679 and 680: E.9. PROJECTION ON CONVEX SET 679 P
- Page 681 and 682: E.9. PROJECTION ON CONVEX SET 681 E
- Page 683 and 684: E.9. PROJECTION ON CONVEX SET 683 T
- Page 685 and 686: E.9. PROJECTION ON CONVEX SET 685
- Page 687 and 688: E.10. ALTERNATING PROJECTION 687 E.
- Page 689 and 690: E.10. ALTERNATING PROJECTION 689 b
- Page 691 and 692: E.10. ALTERNATING PROJECTION 691 a
- Page 693 and 694: E.10. ALTERNATING PROJECTION 693 (a
- Page 695 and 696: E.10. ALTERNATING PROJECTION 695 wh
- Page 697 and 698: E.10. ALTERNATING PROJECTION 697 E.
- Page 699 and 700: E.10. ALTERNATING PROJECTION 699 10
- Page 701 and 702: E.10. ALTERNATING PROJECTION 701 E.
- Page 703 and 704: E.10. ALTERNATING PROJECTION 703 E
- Page 705 and 706: Appendix F Notation and a few defin
- Page 707 and 708: 707 a.i. c.i. l.i. w.r.t affinely i
- Page 709 and 710: 709 is or ← → t → 0 + as in
E.6. VECTORIZATION INTERPRETATION, 661<br />
E.5.0.0.8 Example. Projecting on hyperplane, halfspace, slab.<br />
Given the hyperplane representation having b ∈ R and nonzero normal<br />
a∈ R m ∂H = {y | a T y = b} ⊂ R m (105)<br />
the orthogonal projection of any point x∈ R m on that hyperplane is<br />
Px = x − a(a T a) −1 (a T x − b) (1833)<br />
Orthogonal projection of x on the halfspace parametrized by b ∈ R and<br />
nonzero normal a∈ R m H − = {y | a T y ≤ b} ⊂ R m (97)<br />
is the point<br />
Px = x − a(a T a) −1 max{0, a T x − b} (1834)<br />
Orthogonal projection of x on the convex slab (Figure 11), for c < b<br />
is the point [123,5.1]<br />
B ∆ = {y | c ≤ a T y ≤ b} ⊂ R m (1835)<br />
Px = x − a(a T a) −1( max{0, a T x − b} − max{0, c − a T x} ) (1836)<br />
<br />
E.6 Vectorization interpretation,<br />
projection on a matrix<br />
E.6.1<br />
Nonorthogonal projection on a vector<br />
Nonorthogonal projection of vector x on the range of vector y is<br />
accomplished using a normalized dyad P 0 (B.1); videlicet,<br />
〈z,x〉<br />
〈z,y〉 y = zT x<br />
z T y y = yzT<br />
z T y x = ∆ P 0 x (1837)<br />
where 〈z,x〉/〈z,y〉 is the coefficient of projection on y . Because P0 2 =P 0<br />
and R(P 0 )= R(y) , rank-one matrix P 0 is a nonorthogonal projector dyad<br />
projecting on R(y). Direction of nonorthogonal projection is orthogonal<br />
to z ; id est,<br />
P 0 x − x ⊥ R(P0 T ) (1838)