v2010.10.26 - Convex Optimization
v2010.10.26 - Convex Optimization v2010.10.26 - Convex Optimization
636 APPENDIX B. SIMPLE MATRICESProof. Figure 156 shows the four fundamental subspaces for the dyad.Linear operator Ψ : R N →R M provides a map between vector spaces thatremain distinct when M =N ;u ∈ R(uv T )u ∈ N(uv T ) ⇔ v T u = 0R(uv T ) ∩ N(uv T ) = ∅(1615)B.1.0.1rank-one modificationFor A∈ R M×N , x∈ R N , y ∈ R M , and y T Ax≠0 [206,2.1]( )rank A − AxyT A= rank(A) − 1 (1616)y T AxIf A∈ R N×N is any nonsingular matrix and 1 + v T A −1 u ≠ 0, then[221, App.6] [393,2.3 prob.16] [153,4.11.2] (Sherman-Morrison)(A + uv T ) −1 = A −1 − A−1 uv T A −11 + v T A −1 u(1617)B.1.0.2dyad symmetryIn the specific circumstance that v = u , then uu T ∈ R N×N is symmetric,rank-one, and positive semidefinite having exactly N −1 0-eigenvalues. Infact, (Theorem A.3.1.0.7)uv T ≽ 0 ⇔ v = u (1618)and the remaining eigenvalue is almost always positive;λ = u T u = tr(uu T ) > 0 unless u=0 (1619)The matrix [ Ψ uu T 1](1620)for example, is rank-1 positive semidefinite if and only if Ψ = uu T .
B.1. RANK-ONE MATRIX (DYAD) 637B.1.1Dyad independenceNow we consider a sum of dyads like (1606) as encountered in diagonalizationand singular value decomposition:( k∑)k∑R s i wiT = R ( )k∑s i wiT = R(s i ) ⇐ w i ∀i are l.i. (1621)i=1i=1range of summation is the vector sum of ranges. B.3 (Theorem B.1.1.1.1)Under the assumption the dyads are linearly independent (l.i.), then vectorsums are unique (p.772): for {w i } l.i. and {s i } l.i.( k∑R s i wiTi=1)i=1= R ( s 1 w T 1)⊕ ... ⊕ R(sk w T k)= R(s1 ) ⊕ ... ⊕ R(s k ) (1622)B.1.1.0.1 Definition. Linearly independent dyads. [211, p.29 thm.11][339, p.2] The set of k dyads{si w T i | i=1... k } (1623)where s i ∈ C M and w i ∈ C N , is said to be linearly independent iff()k∑rank SW T s i wiT = k (1624)i=1where S [s 1 · · · s k ] ∈ C M×k and W [w 1 · · · w k ] ∈ C N×k .△Dyad independence does not preclude existence of a nullspace N(SW T ) ,as defined, nor does it imply SW T were full-rank. In absence of assumptionof independence, generally, rankSW T ≤ k . Conversely, any rank-k matrixcan be written in the form SW T by singular value decomposition. (A.6)B.1.1.0.2 Theorem. Linearly independent (l.i.) dyads.Vectors {s i ∈ C M , i=1... k} are l.i. and vectors {w i ∈ C N , i=1... k} arel.i. if and only if dyads {s i wi T ∈ C M×N , i=1... k} are l.i.⋄B.3 Move of range R to inside summation depends on linear independence of {w i }.
- Page 585 and 586: 7.3. THIRD PREVALENT PROBLEM: 585Gi
- Page 587 and 588: 7.3. THIRD PREVALENT PROBLEM: 587Op
- Page 589 and 590: 7.4. CONCLUSION 589filtering, multi
- Page 591 and 592: Appendix ALinear algebraA.1 Main-di
- Page 593 and 594: A.1. MAIN-DIAGONAL δ OPERATOR, λ
- Page 595 and 596: A.1. MAIN-DIAGONAL δ OPERATOR, λ
- Page 597 and 598: A.2. SEMIDEFINITENESS: DOMAIN OF TE
- Page 599 and 600: A.3. PROPER STATEMENTS 599(AB) T
- Page 601 and 602: A.3. PROPER STATEMENTS 601A.3.1Semi
- Page 603 and 604: A.3. PROPER STATEMENTS 603For A dia
- Page 605 and 606: A.3. PROPER STATEMENTS 605Diagonali
- Page 607 and 608: A.3. PROPER STATEMENTS 607For A,B
- Page 609 and 610: A.3. PROPER STATEMENTS 609When B is
- Page 611 and 612: A.4. SCHUR COMPLEMENT 611A.4 Schur
- Page 613 and 614: A.4. SCHUR COMPLEMENT 613A.4.0.0.3
- Page 615 and 616: A.4. SCHUR COMPLEMENT 615From Corol
- Page 617 and 618: A.5. EIGENVALUE DECOMPOSITION 617wh
- Page 619 and 620: A.5. EIGENVALUE DECOMPOSITION 619A.
- Page 621 and 622: A.6. SINGULAR VALUE DECOMPOSITION,
- Page 623 and 624: A.6. SINGULAR VALUE DECOMPOSITION,
- Page 625 and 626: A.7. ZEROS 625A.6.5SVD of symmetric
- Page 627 and 628: A.7. ZEROS 627(Transpose.)Likewise,
- Page 629 and 630: A.7. ZEROS 629For X,A∈ S M +[34,2
- Page 631 and 632: A.7. ZEROS 631A.7.5.0.1 Proposition
- Page 633 and 634: Appendix BSimple matricesMathematic
- Page 635: B.1. RANK-ONE MATRIX (DYAD) 635R(v)
- Page 639 and 640: B.2. DOUBLET 639R([u v ])R(Π)= R([
- Page 641 and 642: B.3. ELEMENTARY MATRIX 641has N −
- Page 643 and 644: B.4. AUXILIARY V -MATRICES 643is an
- Page 645 and 646: B.4. AUXILIARY V -MATRICES 64514. [
- Page 647 and 648: B.5. ORTHOGONAL MATRIX 647Given X
- Page 649 and 650: B.5. ORTHOGONAL MATRIX 649Figure 15
- Page 651 and 652: B.5. ORTHOGONAL MATRIX 651which is
- Page 653 and 654: Appendix CSome analytical optimal r
- Page 655 and 656: C.2. TRACE, SINGULAR AND EIGEN VALU
- Page 657 and 658: C.2. TRACE, SINGULAR AND EIGEN VALU
- Page 659 and 660: C.2. TRACE, SINGULAR AND EIGEN VALU
- Page 661 and 662: C.3. ORTHOGONAL PROCRUSTES PROBLEM
- Page 663 and 664: C.4. TWO-SIDED ORTHOGONAL PROCRUSTE
- Page 665 and 666: C.4. TWO-SIDED ORTHOGONAL PROCRUSTE
- Page 667 and 668: C.4. TWO-SIDED ORTHOGONAL PROCRUSTE
- Page 669 and 670: Appendix DMatrix calculusFrom too m
- Page 671 and 672: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 673 and 674: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 675 and 676: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 677 and 678: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 679 and 680: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 681 and 682: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 683 and 684: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 685 and 686: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
636 APPENDIX B. SIMPLE MATRICESProof. Figure 156 shows the four fundamental subspaces for the dyad.Linear operator Ψ : R N →R M provides a map between vector spaces thatremain distinct when M =N ;u ∈ R(uv T )u ∈ N(uv T ) ⇔ v T u = 0R(uv T ) ∩ N(uv T ) = ∅(1615)B.1.0.1rank-one modificationFor A∈ R M×N , x∈ R N , y ∈ R M , and y T Ax≠0 [206,2.1]( )rank A − AxyT A= rank(A) − 1 (1616)y T AxIf A∈ R N×N is any nonsingular matrix and 1 + v T A −1 u ≠ 0, then[221, App.6] [393,2.3 prob.16] [153,4.11.2] (Sherman-Morrison)(A + uv T ) −1 = A −1 − A−1 uv T A −11 + v T A −1 u(1617)B.1.0.2dyad symmetryIn the specific circumstance that v = u , then uu T ∈ R N×N is symmetric,rank-one, and positive semidefinite having exactly N −1 0-eigenvalues. Infact, (Theorem A.3.1.0.7)uv T ≽ 0 ⇔ v = u (1618)and the remaining eigenvalue is almost always positive;λ = u T u = tr(uu T ) > 0 unless u=0 (1619)The matrix [ Ψ uu T 1](1620)for example, is rank-1 positive semidefinite if and only if Ψ = uu T .