v2010.10.26 - Convex Optimization

v2010.10.26 - Convex Optimization v2010.10.26 - Convex Optimization

convexoptimization.com
from convexoptimization.com More from this publisher
12.07.2015 Views

574 CHAPTER 7. PROXIMITY PROBLEMS7.2.2.4 Rank minimization heuristic beyond convex envelopeFazel, Hindi, & Boyd [136] [387] [137] propose a rank heuristic more potentthan trace (1365) for problems of rank minimization;rankY ← log det(Y +εI) (1371)the concave surrogate function log det in place of quasiconcave rankY(2.9.2.9.2) when Y ∈ S n + is variable and where ε is a small positive constant.They propose minimization of the surrogate by substituting a sequencecomprising infima of a linearized surrogate about the current estimate Y i ;id est, from the first-order Taylor series expansion about Y i on some openinterval of ‖Y ‖ 2 (D.1.7)log det(Y + εI) ≈ log det(Y i + εI) + tr ( (Y i + εI) −1 (Y − Y i ) ) (1372)we make the surrogate sequence of infima over bounded convex feasible set Cwhere, for i = 0...arg inf rankY ← lim Y i+1 (1373)Y ∈ C i→∞Y i+1 = arg infY ∈ C tr( (Y i + εI) −1 Y ) (1374)a matrix analogue to the reweighting scheme disclosed in [207,4.11.3].Choosing Y 0 =I , the first step becomes equivalent to finding the infimum oftrY ; the trace rank-heuristic (1365). The intuition underlying (1374) is thenew term in the argument of trace; specifically, (Y i + εI) −1 weights Y so thatrelatively small eigenvalues of Y found by the infimum are made even smaller.To see that, substitute the nonincreasingly ordered diagonalizationsY i + εI Q(Λ + εI)Q TY UΥU Tinto (1374). Then from (1701) we have,inf δ((Λ + εI) −1 ) T δ(Υ) =Υ∈U ⋆T CU ⋆≤infΥ∈U T CU(a)(b)inf tr ( (Λ + εI) −1 R T ΥR )R T =R −1inf tr((Y i + εI) −1 Y )Y ∈ C(1375)(1376)where R Q T U in U on the set of orthogonal matrices is a bijection. Therole of ε is, therefore, to limit maximum weight; the smallest entry on themain diagonal of Υ gets the largest weight.

7.2. SECOND PREVALENT PROBLEM: 5757.2.2.5 Applying log det rank-heuristic to Problem 2When the log det rank-heuristic is inserted into Problem 2, problem (1370)becomes the problem sequence in iminimizeD , Y , κsubject toκρ + 2 tr(V Y V )[ ]djl y jl≽ 0 ,y jlh 2 jll > j = 1... N −1− tr((−V D i V + εI) −1 V DV ) ≤ κρ(1377)Y ∈ S N hD ∈ EDM Nwhere D i+1 D ⋆ ∈ EDM N and D 0 11 T − I .7.2.2.6 Tightening this log det rank-heuristicLike the trace method, this log det technique for constraining rank offersno provision for meeting a predetermined upper bound ρ . Yet sinceeigenvalues are simply determined, λ(Y i + εI)=δ(Λ + εI) , we may certainlyforce selected weights to ε −1 by manipulating diagonalization (1375a).Empirically we find this sometimes leads to better results, although affinedimension of a solution cannot be guaranteed.7.2.2.7 Cumulative summary of rank heuristicsWe have studied a perturbation method of rank reduction in4.3 as well asthe trace heuristic (convex envelope method7.2.2.1.1) and log det heuristicin7.2.2.4. There is another good contemporary method called LMIRank[285] based on alternating projection (E.10). 7.177.2.2.7.1 Example. Unidimensional scaling.We apply the convex iteration method from4.4.1 to numerically solve aninstance of Problem 2; a method empirically superior to the foregoing convexenvelope and log det heuristics for rank regularization and enforcing affinedimension.7.17 that does not solve the ball packing problem presented in5.4.2.3.4.

7.2. SECOND PREVALENT PROBLEM: 5757.2.2.5 Applying log det rank-heuristic to Problem 2When the log det rank-heuristic is inserted into Problem 2, problem (1370)becomes the problem sequence in iminimizeD , Y , κsubject toκρ + 2 tr(V Y V )[ ]djl y jl≽ 0 ,y jlh 2 jll > j = 1... N −1− tr((−V D i V + εI) −1 V DV ) ≤ κρ(1377)Y ∈ S N hD ∈ EDM Nwhere D i+1 D ⋆ ∈ EDM N and D 0 11 T − I .7.2.2.6 Tightening this log det rank-heuristicLike the trace method, this log det technique for constraining rank offersno provision for meeting a predetermined upper bound ρ . Yet sinceeigenvalues are simply determined, λ(Y i + εI)=δ(Λ + εI) , we may certainlyforce selected weights to ε −1 by manipulating diagonalization (1375a).Empirically we find this sometimes leads to better results, although affinedimension of a solution cannot be guaranteed.7.2.2.7 Cumulative summary of rank heuristicsWe have studied a perturbation method of rank reduction in4.3 as well asthe trace heuristic (convex envelope method7.2.2.1.1) and log det heuristicin7.2.2.4. There is another good contemporary method called LMIRank[285] based on alternating projection (E.10). 7.177.2.2.7.1 Example. Unidimensional scaling.We apply the convex iteration method from4.4.1 to numerically solve aninstance of Problem 2; a method empirically superior to the foregoing convexenvelope and log det heuristics for rank regularization and enforcing affinedimension.7.17 that does not solve the ball packing problem presented in5.4.2.3.4.

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!