v2010.10.26 - Convex Optimization
v2010.10.26 - Convex Optimization v2010.10.26 - Convex Optimization
574 CHAPTER 7. PROXIMITY PROBLEMS7.2.2.4 Rank minimization heuristic beyond convex envelopeFazel, Hindi, & Boyd [136] [387] [137] propose a rank heuristic more potentthan trace (1365) for problems of rank minimization;rankY ← log det(Y +εI) (1371)the concave surrogate function log det in place of quasiconcave rankY(2.9.2.9.2) when Y ∈ S n + is variable and where ε is a small positive constant.They propose minimization of the surrogate by substituting a sequencecomprising infima of a linearized surrogate about the current estimate Y i ;id est, from the first-order Taylor series expansion about Y i on some openinterval of ‖Y ‖ 2 (D.1.7)log det(Y + εI) ≈ log det(Y i + εI) + tr ( (Y i + εI) −1 (Y − Y i ) ) (1372)we make the surrogate sequence of infima over bounded convex feasible set Cwhere, for i = 0...arg inf rankY ← lim Y i+1 (1373)Y ∈ C i→∞Y i+1 = arg infY ∈ C tr( (Y i + εI) −1 Y ) (1374)a matrix analogue to the reweighting scheme disclosed in [207,4.11.3].Choosing Y 0 =I , the first step becomes equivalent to finding the infimum oftrY ; the trace rank-heuristic (1365). The intuition underlying (1374) is thenew term in the argument of trace; specifically, (Y i + εI) −1 weights Y so thatrelatively small eigenvalues of Y found by the infimum are made even smaller.To see that, substitute the nonincreasingly ordered diagonalizationsY i + εI Q(Λ + εI)Q TY UΥU Tinto (1374). Then from (1701) we have,inf δ((Λ + εI) −1 ) T δ(Υ) =Υ∈U ⋆T CU ⋆≤infΥ∈U T CU(a)(b)inf tr ( (Λ + εI) −1 R T ΥR )R T =R −1inf tr((Y i + εI) −1 Y )Y ∈ C(1375)(1376)where R Q T U in U on the set of orthogonal matrices is a bijection. Therole of ε is, therefore, to limit maximum weight; the smallest entry on themain diagonal of Υ gets the largest weight.
7.2. SECOND PREVALENT PROBLEM: 5757.2.2.5 Applying log det rank-heuristic to Problem 2When the log det rank-heuristic is inserted into Problem 2, problem (1370)becomes the problem sequence in iminimizeD , Y , κsubject toκρ + 2 tr(V Y V )[ ]djl y jl≽ 0 ,y jlh 2 jll > j = 1... N −1− tr((−V D i V + εI) −1 V DV ) ≤ κρ(1377)Y ∈ S N hD ∈ EDM Nwhere D i+1 D ⋆ ∈ EDM N and D 0 11 T − I .7.2.2.6 Tightening this log det rank-heuristicLike the trace method, this log det technique for constraining rank offersno provision for meeting a predetermined upper bound ρ . Yet sinceeigenvalues are simply determined, λ(Y i + εI)=δ(Λ + εI) , we may certainlyforce selected weights to ε −1 by manipulating diagonalization (1375a).Empirically we find this sometimes leads to better results, although affinedimension of a solution cannot be guaranteed.7.2.2.7 Cumulative summary of rank heuristicsWe have studied a perturbation method of rank reduction in4.3 as well asthe trace heuristic (convex envelope method7.2.2.1.1) and log det heuristicin7.2.2.4. There is another good contemporary method called LMIRank[285] based on alternating projection (E.10). 7.177.2.2.7.1 Example. Unidimensional scaling.We apply the convex iteration method from4.4.1 to numerically solve aninstance of Problem 2; a method empirically superior to the foregoing convexenvelope and log det heuristics for rank regularization and enforcing affinedimension.7.17 that does not solve the ball packing problem presented in5.4.2.3.4.
- Page 523 and 524: 6.7. A GEOMETRY OF COMPLETION 523(b
- Page 525 and 526: 6.7. A GEOMETRY OF COMPLETION 525[3
- Page 527 and 528: 6.7. A GEOMETRY OF COMPLETION 527wh
- Page 529 and 530: 6.8. DUAL EDM CONE 529to the geomet
- Page 531 and 532: 6.8. DUAL EDM CONE 531Proof. First,
- Page 533 and 534: 6.8. DUAL EDM CONE 533EDM 2 = S 2 h
- Page 535 and 536: 6.8. DUAL EDM CONE 535therefore the
- Page 537 and 538: 6.8. DUAL EDM CONE 537Elegance of t
- Page 539 and 540: 6.8. DUAL EDM CONE 5396.8.1.5 Affin
- Page 541 and 542: 6.8. DUAL EDM CONE 5416.8.1.7 Schoe
- Page 543 and 544: 6.8. DUAL EDM CONE 5430dvec rel ∂
- Page 545 and 546: 6.10. POSTSCRIPT 5456.10 Postscript
- Page 547 and 548: Chapter 7Proximity problemsIn the
- Page 549 and 550: 549project on the subspace, then pr
- Page 551 and 552: 551HS N h0EDM NK = S N h ∩ R N×N
- Page 553 and 554: 5537.0.3 Problem approachProblems t
- Page 555 and 556: 7.1. FIRST PREVALENT PROBLEM: 555fi
- Page 557 and 558: 7.1. FIRST PREVALENT PROBLEM: 5577.
- Page 559 and 560: 7.1. FIRST PREVALENT PROBLEM: 559di
- Page 561 and 562: 7.1. FIRST PREVALENT PROBLEM: 5617.
- Page 563 and 564: 7.1. FIRST PREVALENT PROBLEM: 563wh
- Page 565 and 566: 7.1. FIRST PREVALENT PROBLEM: 565Th
- Page 567 and 568: 7.2. SECOND PREVALENT PROBLEM: 567O
- Page 569 and 570: 7.2. SECOND PREVALENT PROBLEM: 569S
- Page 571 and 572: 7.2. SECOND PREVALENT PROBLEM: 571r
- Page 573: 7.2. SECOND PREVALENT PROBLEM: 573w
- Page 577 and 578: 7.2. SECOND PREVALENT PROBLEM: 577a
- Page 579 and 580: 7.3. THIRD PREVALENT PROBLEM: 579is
- Page 581 and 582: 7.3. THIRD PREVALENT PROBLEM: 581We
- Page 583 and 584: 7.3. THIRD PREVALENT PROBLEM: 583su
- Page 585 and 586: 7.3. THIRD PREVALENT PROBLEM: 585Gi
- Page 587 and 588: 7.3. THIRD PREVALENT PROBLEM: 587Op
- Page 589 and 590: 7.4. CONCLUSION 589filtering, multi
- Page 591 and 592: Appendix ALinear algebraA.1 Main-di
- Page 593 and 594: A.1. MAIN-DIAGONAL δ OPERATOR, λ
- Page 595 and 596: A.1. MAIN-DIAGONAL δ OPERATOR, λ
- Page 597 and 598: A.2. SEMIDEFINITENESS: DOMAIN OF TE
- Page 599 and 600: A.3. PROPER STATEMENTS 599(AB) T
- Page 601 and 602: A.3. PROPER STATEMENTS 601A.3.1Semi
- Page 603 and 604: A.3. PROPER STATEMENTS 603For A dia
- Page 605 and 606: A.3. PROPER STATEMENTS 605Diagonali
- Page 607 and 608: A.3. PROPER STATEMENTS 607For A,B
- Page 609 and 610: A.3. PROPER STATEMENTS 609When B is
- Page 611 and 612: A.4. SCHUR COMPLEMENT 611A.4 Schur
- Page 613 and 614: A.4. SCHUR COMPLEMENT 613A.4.0.0.3
- Page 615 and 616: A.4. SCHUR COMPLEMENT 615From Corol
- Page 617 and 618: A.5. EIGENVALUE DECOMPOSITION 617wh
- Page 619 and 620: A.5. EIGENVALUE DECOMPOSITION 619A.
- Page 621 and 622: A.6. SINGULAR VALUE DECOMPOSITION,
- Page 623 and 624: A.6. SINGULAR VALUE DECOMPOSITION,
7.2. SECOND PREVALENT PROBLEM: 5757.2.2.5 Applying log det rank-heuristic to Problem 2When the log det rank-heuristic is inserted into Problem 2, problem (1370)becomes the problem sequence in iminimizeD , Y , κsubject toκρ + 2 tr(V Y V )[ ]djl y jl≽ 0 ,y jlh 2 jll > j = 1... N −1− tr((−V D i V + εI) −1 V DV ) ≤ κρ(1377)Y ∈ S N hD ∈ EDM Nwhere D i+1 D ⋆ ∈ EDM N and D 0 11 T − I .7.2.2.6 Tightening this log det rank-heuristicLike the trace method, this log det technique for constraining rank offersno provision for meeting a predetermined upper bound ρ . Yet sinceeigenvalues are simply determined, λ(Y i + εI)=δ(Λ + εI) , we may certainlyforce selected weights to ε −1 by manipulating diagonalization (1375a).Empirically we find this sometimes leads to better results, although affinedimension of a solution cannot be guaranteed.7.2.2.7 Cumulative summary of rank heuristicsWe have studied a perturbation method of rank reduction in4.3 as well asthe trace heuristic (convex envelope method7.2.2.1.1) and log det heuristicin7.2.2.4. There is another good contemporary method called LMIRank[285] based on alternating projection (E.10). 7.177.2.2.7.1 Example. Unidimensional scaling.We apply the convex iteration method from4.4.1 to numerically solve aninstance of Problem 2; a method empirically superior to the foregoing convexenvelope and log det heuristics for rank regularization and enforcing affinedimension.7.17 that does not solve the ball packing problem presented in5.4.2.3.4.