v2007.09.13 - Convex Optimization
v2007.09.13 - Convex Optimization v2007.09.13 - Convex Optimization
464 CHAPTER 7. PROXIMITY PROBLEMSTo see that, substitute the nonincreasingly ordered diagonalizationsY i + εI ∆ = Q(Λ + εI)Q TY ∆ = UΥU T(a)(b)(1175)into (1174). Then from (1476) we have,inf δ((Λ + εI) −1 ) T δ(Υ) =Υ∈U ⋆T CU ⋆≤infΥ∈U T CUinf tr ( (Λ + εI) −1 R T ΥR )R T =R −1inf tr((Y i + εI) −1 Y )Y ∈ C(1176)where R = ∆ Q T U in U on the set of orthogonal matrices is a bijection. Therole of ε is, therefore, to limit the maximum weight; the smallest entry onthe main diagonal of Υ gets the largest weight.7.2.2.5 Applying log det rank-heuristic to Problem 2When the log det rank-heuristic is inserted into Problem 2, problem (1170)becomes the problem sequence in iminimizeD , Y , κsubject toκ ρ + 2 tr(V Y V )[ ]djl y jl≽ 0 ,y jlh 2 jll > j = 1... N −1− tr((−V D i V + εI) −1 V DV ) ≤ κ ρ(1177)Y ∈ S N hD ∈ EDM Nwhere D i+1 ∆ =D ⋆ ∈ EDM N , and D 0 ∆ =11 T − I .7.2.2.6 Tightening this log det rank-heuristicLike the trace method, this log det technique for constraining rank offersno provision for meeting a predetermined upper bound ρ . Yet since theeigenvalues of the sum are simply determined, λ(Y i + εI) = δ(Λ + εI) , wemay certainly force selected weights to ε −1 by manipulating diagonalization(1175a). Empirically we find this sometimes leads to better results, althoughaffine dimension of a solution cannot be guaranteed.
7.2. SECOND PREVALENT PROBLEM: 4657.2.2.7 Cumulative summary of rank heuristicsWe have studied the perturbation method of rank reduction in4.3, aswell as the trace heuristic (convex envelope method7.2.2.1.1) and log detheuristic in7.2.2.4. There is another good contemporary method calledLMIRank [209] based on alternating projection (E.10) that does not solvethe ball packing problem presented in5.4.2.2.3, so it is not evaluated furtherherein. None of these exceed performance of the convex iteration method forconstraining rank developed in4.4:7.2.2.7.1 Example. Rank regularization enforcing affine dimension.We apply the convex iteration method from4.4.1 to numerically solve aninstance of Problem 2; a method empirically superior to the foregoing convexenvelope and log det heuristics.Unidimensional scaling, [70] a historically practical application ofmultidimensional scaling (5.12), entails solution of an optimization problemhaving local minima whose multiplicity varies as the factorial of point-listcardinality. Geometrically, it means finding a list constrained to lie in oneaffine dimension. In terms of point list, the nonconvex problem is: givennonnegative symmetric matrix H = [h ij ] ∈ S N ∩ R N×N+ (1159) whose entriesh ij are all known, (1110)minimize{x i ∈R}N∑(‖x i − x j ‖ − h ij ) 2 (1178)i , j=1called a raw stress problem [39, p.34] which has an implicit constraint ondimensional embedding of points {x i ∈ R , i = 1... N}. This problem hasproven NP-hard; e.g., [52].As always, we first transform variables to distance-square D ∈ S N h ; so webegin with convex problem (1161) on page 459minimize − tr(V (D − 2Y )V )D , Y[ ]dij y ijsubject to≽ 0 ,y ijh 2 ijY ∈ S N hD ∈ EDM NrankV T N DV N = 1j > i = 1... N −1(1179)
- Page 413 and 414: 6.6. CORRESPONDENCE TO PSD CONE S N
- Page 415 and 416: 6.7. VECTORIZATION & PROJECTION INT
- Page 417 and 418: 6.7. VECTORIZATION & PROJECTION INT
- Page 419 and 420: 6.8. DUAL EDM CONE 419When the Fins
- Page 421 and 422: 6.8. DUAL EDM CONE 421Proof. First,
- Page 423 and 424: 6.8. DUAL EDM CONE 423EDM 2 = S 2 h
- Page 425 and 426: 6.8. DUAL EDM CONE 425whose veracit
- Page 427 and 428: 6.8. DUAL EDM CONE 4276.8.1.3.1 Exe
- Page 429 and 430: 6.8. DUAL EDM CONE 429has dual affi
- Page 431 and 432: 6.8. DUAL EDM CONE 4316.8.1.7 Schoe
- Page 433 and 434: 6.9. THEOREM OF THE ALTERNATIVE 433
- Page 435 and 436: 6.10. POSTSCRIPT 435When D is an ED
- Page 437 and 438: Chapter 7Proximity problemsIn summa
- Page 439 and 440: In contrast, order of projection on
- Page 441 and 442: 441HS N h0EDM NK = S N h ∩ R N×N
- Page 443 and 444: 4437.0.3 Problem approachProblems t
- Page 445 and 446: 7.1. FIRST PREVALENT PROBLEM: 445fi
- Page 447 and 448: 7.1. FIRST PREVALENT PROBLEM: 4477.
- Page 449 and 450: 7.1. FIRST PREVALENT PROBLEM: 449di
- Page 451 and 452: 7.1. FIRST PREVALENT PROBLEM: 4517.
- Page 453 and 454: 7.1. FIRST PREVALENT PROBLEM: 453wh
- Page 455 and 456: 7.1. FIRST PREVALENT PROBLEM: 455Th
- Page 457 and 458: 7.2. SECOND PREVALENT PROBLEM: 457O
- Page 459 and 460: 7.2. SECOND PREVALENT PROBLEM: 459S
- Page 461 and 462: 7.2. SECOND PREVALENT PROBLEM: 461r
- Page 463: 7.2. SECOND PREVALENT PROBLEM: 463c
- Page 467 and 468: 7.3. THIRD PREVALENT PROBLEM: 467fo
- Page 469 and 470: 7.3. THIRD PREVALENT PROBLEM: 469a
- Page 471 and 472: 7.3. THIRD PREVALENT PROBLEM: 4717.
- Page 473 and 474: 7.3. THIRD PREVALENT PROBLEM: 4737.
- Page 475 and 476: 7.3. THIRD PREVALENT PROBLEM: 475Ou
- Page 478 and 479: 478 CHAPTER 7. PROXIMITY PROBLEMSth
- Page 480 and 481: 480 APPENDIX A. LINEAR ALGEBRAA.1.1
- Page 482 and 483: 482 APPENDIX A. LINEAR ALGEBRAA.1.2
- Page 484 and 485: 484 APPENDIX A. LINEAR ALGEBRAonly
- Page 486 and 487: 486 APPENDIX A. LINEAR ALGEBRA(AB)
- Page 488 and 489: 488 APPENDIX A. LINEAR ALGEBRAA.3.1
- Page 490 and 491: 490 APPENDIX A. LINEAR ALGEBRAFor A
- Page 492 and 493: 492 APPENDIX A. LINEAR ALGEBRADiago
- Page 494 and 495: 494 APPENDIX A. LINEAR ALGEBRAFor A
- Page 496 and 497: 496 APPENDIX A. LINEAR ALGEBRAA.3.1
- Page 498 and 499: 498 APPENDIX A. LINEAR ALGEBRAA.4 S
- Page 500 and 501: 500 APPENDIX A. LINEAR ALGEBRAA.4.0
- Page 502 and 503: 502 APPENDIX A. LINEAR ALGEBRAA.5 e
- Page 504 and 505: 504 APPENDIX A. LINEAR ALGEBRAs i w
- Page 506 and 507: 506 APPENDIX A. LINEAR ALGEBRAA.6.2
- Page 508 and 509: 508 APPENDIX A. LINEAR ALGEBRAΣq 2
- Page 510 and 511: 510 APPENDIX A. LINEAR ALGEBRAA.7 Z
- Page 512 and 513: 512 APPENDIX A. LINEAR ALGEBRAThere
464 CHAPTER 7. PROXIMITY PROBLEMSTo see that, substitute the nonincreasingly ordered diagonalizationsY i + εI ∆ = Q(Λ + εI)Q TY ∆ = UΥU T(a)(b)(1175)into (1174). Then from (1476) we have,inf δ((Λ + εI) −1 ) T δ(Υ) =Υ∈U ⋆T CU ⋆≤infΥ∈U T CUinf tr ( (Λ + εI) −1 R T ΥR )R T =R −1inf tr((Y i + εI) −1 Y )Y ∈ C(1176)where R = ∆ Q T U in U on the set of orthogonal matrices is a bijection. Therole of ε is, therefore, to limit the maximum weight; the smallest entry onthe main diagonal of Υ gets the largest weight.7.2.2.5 Applying log det rank-heuristic to Problem 2When the log det rank-heuristic is inserted into Problem 2, problem (1170)becomes the problem sequence in iminimizeD , Y , κsubject toκ ρ + 2 tr(V Y V )[ ]djl y jl≽ 0 ,y jlh 2 jll > j = 1... N −1− tr((−V D i V + εI) −1 V DV ) ≤ κ ρ(1177)Y ∈ S N hD ∈ EDM Nwhere D i+1 ∆ =D ⋆ ∈ EDM N , and D 0 ∆ =11 T − I .7.2.2.6 Tightening this log det rank-heuristicLike the trace method, this log det technique for constraining rank offersno provision for meeting a predetermined upper bound ρ . Yet since theeigenvalues of the sum are simply determined, λ(Y i + εI) = δ(Λ + εI) , wemay certainly force selected weights to ε −1 by manipulating diagonalization(1175a). Empirically we find this sometimes leads to better results, althoughaffine dimension of a solution cannot be guaranteed.
Change language