v2009.01.01 - Convex Optimization
v2009.01.01 - Convex Optimization v2009.01.01 - Convex Optimization
582 APPENDIX B. SIMPLE MATRICES B.1.1.1 Biorthogonality condition, Range and Nullspace of Sum Dyads characterized by a biorthogonality condition W T S =I are independent; id est, for S ∈ C M×k and W ∈ C N×k , if W T S =I then rank(SW T )=k by the linearly independent dyads theorem because (conferE.1.1) W T S =I ⇒ rankS=rankW =k≤M =N (1512) To see that, we need only show: N(S)=0 ⇔ ∃ B BS =I . B.4 (⇐) Assume BS =I . Then N(BS)=0={x | BSx = 0} ⊇ N(S). (1493) (⇒) If N(S)=0[ then] S must be full-rank skinny-or-square. B ∴ ∃ A,B,C [ S A] = I (id est, [S A] is invertible) ⇒ BS =I . C Left inverse B is given as W T here. Because of reciprocity with S , it immediately follows: N(W)=0 ⇔ ∃ S S T W =I . Dyads produced by diagonalization, for example, are independent because of their inherent biorthogonality. (A.5.1) The converse is generally false; id est, linearly independent dyads are not necessarily biorthogonal. B.1.1.1.1 Theorem. Nullspace and range of dyad sum. Given a sum of dyads represented by SW T where S ∈C M×k and W ∈ C N×k N(SW T ) = N(W T ) ⇐ ∃ B BS = I R(SW T ) = R(S) ⇐ ∃ Z W T Z = I (1513) ⋄ Proof. (⇒) N(SW T )⊇ N(W T ) and R(SW T )⊆ R(S) are obvious. (⇐) Assume the existence of a left inverse B ∈ R k×N and a right inverse Z ∈ R N×k . B.5 N(SW T ) = {x | SW T x = 0} ⊆ {x | BSW T x = 0} = N(W T ) (1514) R(SW T ) = {SW T x | x∈ R N } ⊇ {SW T Zy | Zy ∈ R N } = R(S) (1515) B.4 Left inverse is not unique, in general. B.5 By counter-example, the theorem’s converse cannot be true; e.g., S = W = [1 0].
B.2. DOUBLET 583 R([u v ]) R(Π)= R([u v]) 0 0 N(Π)=v ⊥ ∩ u ⊥ ([ ]) v R N T = R([u v ]) ⊕ N u T v ⊥ ∩ u ⊥ ([ ]) v T N u T ⊕ R([u v ]) = R N Figure 133: Four fundamental subspaces [289,3.6] of a doublet Π = uv T + vu T ∈ S N . Π(x) = (uv T + vu T )x is a linear bijective mapping from R([u v ]) to R([u v ]). B.2 Doublet Consider a sum of two linearly independent square dyads, one a transposition of the other: Π = uv T + vu T = [ u v ] [ ] v T u T = SW T ∈ S N (1516) where u,v ∈ R N . Like the dyad, a doublet can be 0 only when u or v is 0; Π = uv T + vu T = 0 ⇔ u = 0 or v = 0 (1517) By assumption of independence, a nonzero doublet has two nonzero eigenvalues λ 1 ∆ = u T v + ‖uv T ‖ , λ 2 ∆ = u T v − ‖uv T ‖ (1518) where λ 1 > 0 >λ 2 , with corresponding eigenvectors x 1 ∆ = u ‖u‖ + v ‖v‖ , x 2 ∆ = u ‖u‖ − v ‖v‖ (1519) spanning the doublet range. Eigenvalue λ 1 cannot be 0 unless u and v have opposing directions, but that is antithetical since then the dyads would no longer be independent. Eigenvalue λ 2 is 0 if and only if u and v share the same direction, again antithetical. Generally we have λ 1 > 0 and λ 2 < 0, so Π is indefinite.
- Page 531 and 532: 7.3. THIRD PREVALENT PROBLEM: 531 7
- Page 533 and 534: 7.3. THIRD PREVALENT PROBLEM: 533 O
- Page 535 and 536: 7.4. CONCLUSION 535 The rank constr
- Page 537 and 538: Appendix A Linear algebra A.1 Main-
- Page 539 and 540: A.1. MAIN-DIAGONAL δ OPERATOR, λ
- Page 541 and 542: A.1. MAIN-DIAGONAL δ OPERATOR, λ
- Page 543 and 544: A.2. SEMIDEFINITENESS: DOMAIN OF TE
- Page 545 and 546: A.3. PROPER STATEMENTS 545 (AB) T
- Page 547 and 548: A.3. PROPER STATEMENTS 547 A.3.1 Se
- Page 549 and 550: A.3. PROPER STATEMENTS 549 For A di
- Page 551 and 552: A.3. PROPER STATEMENTS 551 Diagonal
- Page 553 and 554: A.3. PROPER STATEMENTS 553 For A,B
- Page 555 and 556: A.3. PROPER STATEMENTS 555 A.3.1.0.
- Page 557 and 558: A.4. SCHUR COMPLEMENT 557 A.4 Schur
- Page 559 and 560: A.4. SCHUR COMPLEMENT 559 A.4.0.0.2
- Page 561 and 562: A.5. EIGEN DECOMPOSITION 561 When B
- Page 563 and 564: A.5. EIGEN DECOMPOSITION 563 dim N(
- Page 565 and 566: A.6. SINGULAR VALUE DECOMPOSITION,
- Page 567 and 568: A.6. SINGULAR VALUE DECOMPOSITION,
- Page 569 and 570: A.7. ZEROS 569 Given symmetric matr
- Page 571 and 572: A.7. ZEROS 571 (TRANSPOSE.) Likewis
- Page 573 and 574: A.7. ZEROS 573 For X,A∈ S M + [31
- Page 575 and 576: A.7. ZEROS 575 A.7.5.0.1 Propositio
- Page 577 and 578: Appendix B Simple matrices Mathemat
- Page 579 and 580: B.1. RANK-ONE MATRIX (DYAD) 579 R(v
- Page 581: B.1. RANK-ONE MATRIX (DYAD) 581 ran
- Page 585 and 586: B.3. ELEMENTARY MATRIX 585 If λ
- Page 587 and 588: B.4. AUXILIARY V -MATRICES 587 the
- Page 589 and 590: B.4. AUXILIARY V -MATRICES 589 18.
- Page 591 and 592: B.5. ORTHOGONAL MATRIX 591 B.5 Orth
- Page 593 and 594: B.5. ORTHOGONAL MATRIX 593 Figure 1
- Page 595 and 596: Appendix C Some analytical optimal
- Page 597 and 598: C.2. TRACE, SINGULAR AND EIGEN VALU
- Page 599 and 600: C.2. TRACE, SINGULAR AND EIGEN VALU
- Page 601 and 602: C.2. TRACE, SINGULAR AND EIGEN VALU
- Page 603 and 604: C.3. ORTHOGONAL PROCRUSTES PROBLEM
- Page 605 and 606: C.4. TWO-SIDED ORTHOGONAL PROCRUSTE
- Page 607 and 608: C.4. TWO-SIDED ORTHOGONAL PROCRUSTE
- Page 609 and 610: Appendix D Matrix calculus From too
- Page 611 and 612: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 613 and 614: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 615 and 616: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 617 and 618: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 619 and 620: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 621 and 622: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 623 and 624: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 625 and 626: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 627 and 628: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 629 and 630: D.1. DIRECTIONAL DERIVATIVE, TAYLOR
- Page 631 and 632: D.2. TABLES OF GRADIENTS AND DERIVA
B.2. DOUBLET 583<br />
R([u v ])<br />
R(Π)= R([u v])<br />
0 0 <br />
N(Π)=v ⊥ ∩ u ⊥<br />
([ ]) v<br />
R N T<br />
= R([u v ]) ⊕ N<br />
u T<br />
v ⊥ ∩ u ⊥<br />
([ ]) v<br />
T<br />
N<br />
u T ⊕ R([u v ]) = R N<br />
Figure 133: Four fundamental subspaces [289,3.6] of a doublet<br />
Π = uv T + vu T ∈ S N . Π(x) = (uv T + vu T )x is a linear bijective mapping<br />
from R([u v ]) to R([u v ]).<br />
B.2 Doublet<br />
Consider a sum of two linearly independent square dyads, one a transposition<br />
of the other:<br />
Π = uv T + vu T = [ u v ] [ ]<br />
v T<br />
u T = SW T ∈ S N (1516)<br />
where u,v ∈ R N . Like the dyad, a doublet can be 0 only when u or v is 0;<br />
Π = uv T + vu T = 0 ⇔ u = 0 or v = 0 (1517)<br />
By assumption of independence, a nonzero doublet has two nonzero<br />
eigenvalues<br />
λ 1 ∆ = u T v + ‖uv T ‖ , λ 2 ∆ = u T v − ‖uv T ‖ (1518)<br />
where λ 1 > 0 >λ 2 , with corresponding eigenvectors<br />
x 1 ∆ = u<br />
‖u‖ + v<br />
‖v‖ , x 2<br />
∆<br />
= u<br />
‖u‖ −<br />
v<br />
‖v‖<br />
(1519)<br />
spanning the doublet range. Eigenvalue λ 1 cannot be 0 unless u and v have<br />
opposing directions, but that is antithetical since then the dyads would no<br />
longer be independent. Eigenvalue λ 2 is 0 if and only if u and v share the<br />
same direction, again antithetical. Generally we have λ 1 > 0 and λ 2 < 0, so<br />
Π is indefinite.