v2007.09.13 - Convex Optimization
v2007.09.13 - Convex Optimization v2007.09.13 - Convex Optimization
538 APPENDIX C. SOME ANALYTICAL OPTIMAL RESULTSthe largest eigenvalue, and v N a normalized eigenvector correspondingto the smallest eigenvalue,v N = arg inf x T Ax (1468)‖x‖=1v 1 = arg sup x T Ax (1469)‖x‖=1For A∈ S N having eigenvalues λ(A)∈ R N , consider the unconstrainednonconvex optimization that is a projection on the rank-1 subset(2.9.2.1) of the boundary of positive semidefinite cone S N + : Definingλ 1 ∆ = maxi{λ(A) i } and corresponding eigenvector v 1minimizex‖xx T − A‖ 2 F = minimize tr(xx T (x T x) − 2Axx T + A T A)x{‖λ(A)‖ 2 , λ 1 ≤ 0=(1470)‖λ(A)‖ 2 − λ 2 1 , λ 1 > 0arg minimizex‖xx T − A‖ 2 F ={0 , λ1 ≤ 0v 1√λ1 , λ 1 > 0(1471)Proof. This is simply the Eckart & Young solution from7.1.2:x ⋆ x ⋆T ={0 , λ1 ≤ 0λ 1 v 1 v T 1 , λ 1 > 0(1472)minimizexGiven nonincreasingly ordered diagonalization A = QΛQ TΛ = δ(λ(A)) (A.5), then (1470) has minimum valuewhere⎧‖QΛQ T ‖ 2 F = ‖δ(Λ)‖2 , λ 1 ≤ 0⎪⎨⎛⎡‖xx T −A‖ 2 F =λ 1 Q ⎜⎢0⎝⎣. ..⎪⎩ ∥0⎤ ⎞ ∥ ∥∥∥∥∥∥2⎡⎥ ⎟⎦− Λ⎠Q T ⎢=⎣∥Fλ 10.0⎤⎥⎦− δ(Λ)∥2(1473), λ 1 > 0
C.2. DIAGONAL, TRACE, SINGULAR AND EIGEN VALUES 539C.2.0.0.1 Exercise. Rank-1 approximation.Given symmetric matrix A∈ S N , prove:v 1 = arg minimize ‖xx T − A‖ 2 Fxsubject to ‖x‖ = 1(1474)where v 1 is a normalized eigenvector of A corresponding to its largesteigenvalue.(Fan) For B ∈ S N whose eigenvalues λ(B)∈ R N are arranged innonincreasing order, and for 1≤k ≤N [9,4.1] [157] [149,4.3.18][267,2] [174,2.1]N∑i=N−k+1λ(B) i = infU∈ R N×kU T U=Ik∑λ(B) i = supi=1U∈ R N×kU T U=Itr(UU T B) = minimizeX∈ S N +tr(XB)subject to X ≼ ItrX = k= maximize (k − N)µ + tr(B − Z)µ∈R , Z∈S N +subject to µI + Z ≽ Btr(UU T B) = maximizeX∈ S N +tr(XB)subject to X ≼ ItrX = k(a)(b)(c)= minimize kµ + trZµ∈R , Z∈S N +subject to µI + Z ≽ B(d)(1475)Given ordered diagonalization B = QΛQ T , (A.5.2) then optimalU for the infimum is U ⋆ = Q(:, N − k+1:N)∈ R N×k whereasU ⋆ = Q(:, 1:k)∈ R N×k for the supremum. In both cases, X ⋆ = U ⋆ U ⋆T .Optimization (a) searches the convex hull of the outer product UU Tof all N ×k orthonormal matrices. (2.3.2.0.1)
- Page 488 and 489: 488 APPENDIX A. LINEAR ALGEBRAA.3.1
- Page 490 and 491: 490 APPENDIX A. LINEAR ALGEBRAFor A
- Page 492 and 493: 492 APPENDIX A. LINEAR ALGEBRADiago
- Page 494 and 495: 494 APPENDIX A. LINEAR ALGEBRAFor A
- Page 496 and 497: 496 APPENDIX A. LINEAR ALGEBRAA.3.1
- Page 498 and 499: 498 APPENDIX A. LINEAR ALGEBRAA.4 S
- Page 500 and 501: 500 APPENDIX A. LINEAR ALGEBRAA.4.0
- Page 502 and 503: 502 APPENDIX A. LINEAR ALGEBRAA.5 e
- Page 504 and 505: 504 APPENDIX A. LINEAR ALGEBRAs i w
- Page 506 and 507: 506 APPENDIX A. LINEAR ALGEBRAA.6.2
- Page 508 and 509: 508 APPENDIX A. LINEAR ALGEBRAΣq 2
- Page 510 and 511: 510 APPENDIX A. LINEAR ALGEBRAA.7 Z
- Page 512 and 513: 512 APPENDIX A. LINEAR ALGEBRAThere
- Page 514 and 515: 514 APPENDIX A. LINEAR ALGEBRAA.7.5
- Page 516 and 517: 516 APPENDIX A. LINEAR ALGEBRA
- Page 518 and 519: 518 APPENDIX B. SIMPLE MATRICESB.1
- Page 520 and 521: 520 APPENDIX B. SIMPLE MATRICESProo
- Page 522 and 523: 522 APPENDIX B. SIMPLE MATRICESB.1.
- Page 524 and 525: 524 APPENDIX B. SIMPLE MATRICESN(u
- Page 526 and 527: 526 APPENDIX B. SIMPLE MATRICESDue
- Page 528 and 529: 528 APPENDIX B. SIMPLE MATRICESB.4.
- Page 530 and 531: 530 APPENDIX B. SIMPLE MATRICEShas
- Page 532 and 533: 532 APPENDIX B. SIMPLE MATRICESFigu
- Page 534 and 535: 534 APPENDIX B. SIMPLE MATRICESB.5.
- Page 536 and 537: 536 APPENDIX C. SOME ANALYTICAL OPT
- Page 540 and 541: 540 APPENDIX C. SOME ANALYTICAL OPT
- Page 542 and 543: 542 APPENDIX C. SOME ANALYTICAL OPT
- Page 544 and 545: 544 APPENDIX C. SOME ANALYTICAL OPT
- Page 546 and 547: 546 APPENDIX C. SOME ANALYTICAL OPT
- Page 548 and 549: 548 APPENDIX C. SOME ANALYTICAL OPT
- Page 550 and 551: 550 APPENDIX D. MATRIX CALCULUSThe
- Page 552 and 553: 552 APPENDIX D. MATRIX CALCULUSGrad
- Page 554 and 555: 554 APPENDIX D. MATRIX CALCULUSBeca
- Page 556 and 557: 556 APPENDIX D. MATRIX CALCULUSwhic
- Page 558 and 559: 558 APPENDIX D. MATRIX CALCULUS⎡
- Page 560 and 561: 560 APPENDIX D. MATRIX CALCULUS→Y
- Page 562 and 563: 562 APPENDIX D. MATRIX CALCULUSD.1.
- Page 564 and 565: 564 APPENDIX D. MATRIX CALCULUSwhic
- Page 566 and 567: 566 APPENDIX D. MATRIX CALCULUSIn t
- Page 568 and 569: 568 APPENDIX D. MATRIX CALCULUSD.1.
- Page 570 and 571: 570 APPENDIX D. MATRIX CALCULUSD.2
- Page 572 and 573: 572 APPENDIX D. MATRIX CALCULUSalge
- Page 574 and 575: 574 APPENDIX D. MATRIX CALCULUStrac
- Page 576 and 577: 576 APPENDIX D. MATRIX CALCULUSD.2.
- Page 578 and 579: 578 APPENDIX D. MATRIX CALCULUS
- Page 580 and 581: 580 APPENDIX E. PROJECTIONThe follo
- Page 582 and 583: 582 APPENDIX E. PROJECTIONFor matri
- Page 584 and 585: 584 APPENDIX E. PROJECTION(⇐) To
- Page 586 and 587: 586 APPENDIX E. PROJECTIONNonorthog
C.2. DIAGONAL, TRACE, SINGULAR AND EIGEN VALUES 539C.2.0.0.1 Exercise. Rank-1 approximation.Given symmetric matrix A∈ S N , prove:v 1 = arg minimize ‖xx T − A‖ 2 Fxsubject to ‖x‖ = 1(1474)where v 1 is a normalized eigenvector of A corresponding to its largesteigenvalue.(Fan) For B ∈ S N whose eigenvalues λ(B)∈ R N are arranged innonincreasing order, and for 1≤k ≤N [9,4.1] [157] [149,4.3.18][267,2] [174,2.1]N∑i=N−k+1λ(B) i = infU∈ R N×kU T U=Ik∑λ(B) i = supi=1U∈ R N×kU T U=Itr(UU T B) = minimizeX∈ S N +tr(XB)subject to X ≼ ItrX = k= maximize (k − N)µ + tr(B − Z)µ∈R , Z∈S N +subject to µI + Z ≽ Btr(UU T B) = maximizeX∈ S N +tr(XB)subject to X ≼ ItrX = k(a)(b)(c)= minimize kµ + trZµ∈R , Z∈S N +subject to µI + Z ≽ B(d)(1475)Given ordered diagonalization B = QΛQ T , (A.5.2) then optimalU for the infimum is U ⋆ = Q(:, N − k+1:N)∈ R N×k whereasU ⋆ = Q(:, 1:k)∈ R N×k for the supremum. In both cases, X ⋆ = U ⋆ U ⋆T .<strong>Optimization</strong> (a) searches the convex hull of the outer product UU Tof all N ×k orthonormal matrices. (2.3.2.0.1)
Change language