v2010.10.26 - Convex Optimization

4.7. CONSTRAINING RANK OF INDEFINITE MATRICES 3894.7.1 rank-constraint midsummaryWe find that this direction matrix idea works well and quite independentlyof desired rank or affine dimension. This idea of direction matrix is goodprincipally because of its simplicity: When confronted with a problemotherwise convex if not for a rank or cardinality constraint, then thatconstraint becomes a linear regularization term in the objective.There exists a common thread through all these Examples; that being,convex iteration with a direction matrix as normal to a linear regularization(a generalization of the well-known trace heuristic). But each problem type(per Example) possesses its own idiosyncrasies that slightly modify how arank-constrained optimal solution is actually obtained: The ball packingproblem in Chapter 5.4.2.3.4, for example, requires a problem sequence ina progressively larger number of balls to find a good initial value for thedirection matrix, whereas many of the examples in the present chapterrequire an initial value of 0. Finding a Boolean solution in Example 4.6.0.0.8requires a procedure to detect stalls, while other problems have no suchrequirement. The combinatorial Procrustes problem in Example 4.6.0.0.3allows use of a known closed-form solution for direction vector when solvedvia rank constraint, but not when solved via cardinality constraint. Someproblems require a careful weighting of the regularization term, whereas otherproblems do not, and so on. It would be nice if there were a universallyapplicable method for constraining rank; one that is less susceptible to quirksof a particular problem type.Poor initialization of the direction matrix from the regularization canlead to an erroneous result. We speculate one reason to be a simple dearthof optimal solutions of desired rank or cardinality; 4.63 an unfortunate choiceof initial search direction leading astray. Ease of solution by convex iterationoccurs when optimal solutions abound. With this speculation in mind, wenow propose a further generalization of convex iteration for constraining rankthat attempts to ameliorate quirks and unify problem types:4.63 Recall that, in Convex Optimization, an optimal solution generally comes from aconvex set of optimal solutions that can be large.