v2010.10.26 - Convex Optimization
v2010.10.26 - Convex Optimization v2010.10.26 - Convex Optimization
554 CHAPTER 7. PROXIMITY PROBLEMSwhere we have made explicit an imposed upper bound ρ on affine dimensionr = rankV T NDV N = rankV DV (1042)that is benign when ρ =N−1 or H were realizable with r ≤ρ. Problems(1310.2) and (1310.3) are Euclidean projections of a vectorized matrix H onan EDM cone (6.3), whereas problems (1310.1) and (1310.4) are Euclideanprojections of a vectorized matrix −VHV on a PSD cone. 7.2 Problem(1310.4) is not posed in the literature because it has limited theoreticalfoundation. 7.3Analytical solution to (1310.1) is known in closed form for any bound ρalthough, as the problem is stated, it is a convex optimization only in the caseρ =N−1. We show in7.1.4 how (1310.1) becomes a convex optimizationproblem for any ρ when transformed to the spectral domain. When expressedas a function of point list in a matrix X as in (1308), problem (1310.2)becomes a variant of what is known in statistics literature as a stress problem.[53, p.34] [103] [354] Problems (1310.2) and (1310.3) are convex optimizationproblems in D for the case ρ =N−1 wherein (1310.3) becomes equivalentto (1309). Even with the rank constraint removed from (1310.2), we will seethat the convex problem remaining inherently minimizes affine dimension.Generally speaking, each problem in (1310) produces a different resultbecause there is no isometry relating them. Of the various auxiliaryV -matrices (B.4), the geometric centering matrix V (913) appears in theliterature most often although V N (897) is the auxiliary matrix naturallyconsequent to Schoenberg’s seminal exposition [312]. Substitution of anyauxiliary matrix or its pseudoinverse into these problems produces anothervalid problem.Substitution of VNT for left-hand V in (1310.1), in particular, produces adifferent result becauseminimize ‖−V (D − H)V ‖ 2 FD(1311)subject to D ∈ EDM N7.2 Because −VHV is orthogonal projection of −H on the geometric center subspace S N c(E.7.2.0.2), problems (1310.1) and (1310.4) may be interpreted as oblique (nonminimumdistance) projections of −H on a positive semidefinite cone.7.3 D ∈ EDM N ⇒ ◦√ D ∈ EDM N , −V ◦√ DV ∈ S N + (5.10)
7.1. FIRST PREVALENT PROBLEM: 555finds D to attain Euclidean distance of vectorized −VHV to the positivesemidefinite cone in ambient isometrically isomorphic R N(N+1)/2 , whereasminimize ‖−VN T(D − H)V N ‖ 2 FD(1312)subject to D ∈ EDM Nattains Euclidean distance of vectorized −VN THV N to the positivesemidefinite cone in isometrically isomorphic subspace R N(N−1)/2 ; quitedifferent projections 7.4 regardless of whether affine dimension is constrained.But substitution of auxiliary matrix VW T (B.4.3) or V † Nyields the sameresult as (1310.1) because V = V W VW T = V N V † N; id est,‖−V (D − H)V ‖ 2 F = ‖−V W V T W (D − H)V W V T W ‖2 F = ‖−V T W (D − H)V W‖ 2 F= ‖−V N V † N (D − H)V N V † N ‖2 F = ‖−V † N (D − H)V N ‖ 2 F(1313)We see no compelling reason to prefer one particular auxiliary V -matrixover another. Each has its own coherent interpretations; e.g.,5.4.2,6.6,B.4.5. Neither can we say any particular problem formulation producesgenerally better results than another. 7.57.1 First prevalent problem:Projection on PSD coneThis first problemminimize ‖−VN T(D − H)V ⎫N ‖ 2 F ⎪⎬Dsubject to rankVN TDV N ≤ ρ Problem 1 (1314)⎪D ∈ EDM N ⎭7.4 The isomorphism T(Y )=V †TN Y V † N onto SN c = {V X V | X ∈ S N } relates the map in(1312) to that in (1311), but is not an isometry.7.5 All four problem formulations (1310) produce identical results when affinedimension r , implicit to a realizable measurement matrix H , does not exceed desiredaffine dimension ρ; because, the optimal objective value will vanish (‖ · ‖ = 0).
- Page 503 and 504: 6.4. EDM DEFINITION IN 11 T 503That
- Page 505 and 506: 6.4. EDM DEFINITION IN 11 T 505N(1
- Page 507 and 508: 6.4. EDM DEFINITION IN 11 T 507Then
- Page 509 and 510: 6.4. EDM DEFINITION IN 11 T 5096.4.
- Page 511 and 512: 6.5. CORRESPONDENCE TO PSD CONE S N
- Page 513 and 514: 6.5. CORRESPONDENCE TO PSD CONE S N
- Page 515 and 516: 6.5. CORRESPONDENCE TO PSD CONE S N
- Page 517 and 518: 6.6. VECTORIZATION & PROJECTION INT
- Page 519 and 520: 6.6. VECTORIZATION & PROJECTION INT
- Page 521 and 522: 6.6. VECTORIZATION & PROJECTION INT
- Page 523 and 524: 6.7. A GEOMETRY OF COMPLETION 523(b
- Page 525 and 526: 6.7. A GEOMETRY OF COMPLETION 525[3
- Page 527 and 528: 6.7. A GEOMETRY OF COMPLETION 527wh
- Page 529 and 530: 6.8. DUAL EDM CONE 529to the geomet
- Page 531 and 532: 6.8. DUAL EDM CONE 531Proof. First,
- Page 533 and 534: 6.8. DUAL EDM CONE 533EDM 2 = S 2 h
- Page 535 and 536: 6.8. DUAL EDM CONE 535therefore the
- Page 537 and 538: 6.8. DUAL EDM CONE 537Elegance of t
- Page 539 and 540: 6.8. DUAL EDM CONE 5396.8.1.5 Affin
- Page 541 and 542: 6.8. DUAL EDM CONE 5416.8.1.7 Schoe
- Page 543 and 544: 6.8. DUAL EDM CONE 5430dvec rel ∂
- Page 545 and 546: 6.10. POSTSCRIPT 5456.10 Postscript
- Page 547 and 548: Chapter 7Proximity problemsIn the
- Page 549 and 550: 549project on the subspace, then pr
- Page 551 and 552: 551HS N h0EDM NK = S N h ∩ R N×N
- Page 553: 5537.0.3 Problem approachProblems t
- Page 557 and 558: 7.1. FIRST PREVALENT PROBLEM: 5577.
- Page 559 and 560: 7.1. FIRST PREVALENT PROBLEM: 559di
- Page 561 and 562: 7.1. FIRST PREVALENT PROBLEM: 5617.
- Page 563 and 564: 7.1. FIRST PREVALENT PROBLEM: 563wh
- Page 565 and 566: 7.1. FIRST PREVALENT PROBLEM: 565Th
- Page 567 and 568: 7.2. SECOND PREVALENT PROBLEM: 567O
- Page 569 and 570: 7.2. SECOND PREVALENT PROBLEM: 569S
- Page 571 and 572: 7.2. SECOND PREVALENT PROBLEM: 571r
- Page 573 and 574: 7.2. SECOND PREVALENT PROBLEM: 573w
- Page 575 and 576: 7.2. SECOND PREVALENT PROBLEM: 5757
- Page 577 and 578: 7.2. SECOND PREVALENT PROBLEM: 577a
- Page 579 and 580: 7.3. THIRD PREVALENT PROBLEM: 579is
- Page 581 and 582: 7.3. THIRD PREVALENT PROBLEM: 581We
- Page 583 and 584: 7.3. THIRD PREVALENT PROBLEM: 583su
- Page 585 and 586: 7.3. THIRD PREVALENT PROBLEM: 585Gi
- Page 587 and 588: 7.3. THIRD PREVALENT PROBLEM: 587Op
- Page 589 and 590: 7.4. CONCLUSION 589filtering, multi
- Page 591 and 592: Appendix ALinear algebraA.1 Main-di
- Page 593 and 594: A.1. MAIN-DIAGONAL δ OPERATOR, λ
- Page 595 and 596: A.1. MAIN-DIAGONAL δ OPERATOR, λ
- Page 597 and 598: A.2. SEMIDEFINITENESS: DOMAIN OF TE
- Page 599 and 600: A.3. PROPER STATEMENTS 599(AB) T
- Page 601 and 602: A.3. PROPER STATEMENTS 601A.3.1Semi
- Page 603 and 604: A.3. PROPER STATEMENTS 603For A dia
554 CHAPTER 7. PROXIMITY PROBLEMSwhere we have made explicit an imposed upper bound ρ on affine dimensionr = rankV T NDV N = rankV DV (1042)that is benign when ρ =N−1 or H were realizable with r ≤ρ. Problems(1310.2) and (1310.3) are Euclidean projections of a vectorized matrix H onan EDM cone (6.3), whereas problems (1310.1) and (1310.4) are Euclideanprojections of a vectorized matrix −VHV on a PSD cone. 7.2 Problem(1310.4) is not posed in the literature because it has limited theoreticalfoundation. 7.3Analytical solution to (1310.1) is known in closed form for any bound ρalthough, as the problem is stated, it is a convex optimization only in the caseρ =N−1. We show in7.1.4 how (1310.1) becomes a convex optimizationproblem for any ρ when transformed to the spectral domain. When expressedas a function of point list in a matrix X as in (1308), problem (1310.2)becomes a variant of what is known in statistics literature as a stress problem.[53, p.34] [103] [354] Problems (1310.2) and (1310.3) are convex optimizationproblems in D for the case ρ =N−1 wherein (1310.3) becomes equivalentto (1309). Even with the rank constraint removed from (1310.2), we will seethat the convex problem remaining inherently minimizes affine dimension.Generally speaking, each problem in (1310) produces a different resultbecause there is no isometry relating them. Of the various auxiliaryV -matrices (B.4), the geometric centering matrix V (913) appears in theliterature most often although V N (897) is the auxiliary matrix naturallyconsequent to Schoenberg’s seminal exposition [312]. Substitution of anyauxiliary matrix or its pseudoinverse into these problems produces anothervalid problem.Substitution of VNT for left-hand V in (1310.1), in particular, produces adifferent result becauseminimize ‖−V (D − H)V ‖ 2 FD(1311)subject to D ∈ EDM N7.2 Because −VHV is orthogonal projection of −H on the geometric center subspace S N c(E.7.2.0.2), problems (1310.1) and (1310.4) may be interpreted as oblique (nonminimumdistance) projections of −H on a positive semidefinite cone.7.3 D ∈ EDM N ⇒ ◦√ D ∈ EDM N , −V ◦√ DV ∈ S N + (5.10)