v2010.10.26 - Convex Optimization

574 CHAPTER 7. PROXIMITY PROBLEMS7.2.2.4 Rank minimization heuristic beyond convex envelopeFazel, Hindi, & Boyd [136] [387] [137] propose a rank heuristic more potentthan trace (1365) for problems of rank minimization;rankY ← log det(Y +εI) (1371)the concave surrogate function log det in place of quasiconcave rankY(2.9.2.9.2) when Y ∈ S n + is variable and where ε is a small positive constant.They propose minimization of the surrogate by substituting a sequencecomprising infima of a linearized surrogate about the current estimate Y i ;id est, from the first-order Taylor series expansion about Y i on some openinterval of ‖Y ‖ 2 (D.1.7)log det(Y + εI) ≈ log det(Y i + εI) + tr ( (Y i + εI) −1 (Y − Y i ) ) (1372)we make the surrogate sequence of infima over bounded convex feasible set Cwhere, for i = 0...arg inf rankY ← lim Y i+1 (1373)Y ∈ C i→∞Y i+1 = arg infY ∈ C tr( (Y i + εI) −1 Y ) (1374)a matrix analogue to the reweighting scheme disclosed in [207,4.11.3].Choosing Y 0 =I , the first step becomes equivalent to finding the infimum oftrY ; the trace rank-heuristic (1365). The intuition underlying (1374) is thenew term in the argument of trace; specifically, (Y i + εI) −1 weights Y so thatrelatively small eigenvalues of Y found by the infimum are made even smaller.To see that, substitute the nonincreasingly ordered diagonalizationsY i + εI Q(Λ + εI)Q TY UΥU Tinto (1374). Then from (1701) we have,inf δ((Λ + εI) −1 ) T δ(Υ) =Υ∈U ⋆T CU ⋆≤infΥ∈U T CU(a)(b)inf tr ( (Λ + εI) −1 R T ΥR )R T =R −1inf tr((Y i + εI) −1 Y )Y ∈ C(1375)(1376)where R Q T U in U on the set of orthogonal matrices is a bijection. Therole of ε is, therefore, to limit maximum weight; the smallest entry on themain diagonal of Υ gets the largest weight.