v2007.09.13 - Convex Optimization

4.4. RANK-CONSTRAINED SEMIDEFINITE PROGRAM 273This linear program sums the n−k smallest entries from vector x . Incontext of problem (645), we want n−k entries of x to sum to zero; id est,we want a globally optimal objective x ⋆T y ⋆ = 0 to vanish. Because all entriesin x must be nonnegative, then n−k entries are themselves zero whenevertheir sum is, and then cardinality of x∈ R n is at most k .Ideally, one wants to solve (645) directly, but contemporary techniques fordoing so are computationally intensive. 4.26 Nevertheless, solving (645) shouldnot be ruled out, assuming an efficient method is discovered or transformationto a convex equivalent can be found.One efficient way to solve (645) is by transforming it to a sequence ofconvex problems:minimize x T yx∈R nsubject to Ax = b(646)x ≽ 0minimize x T yy∈R nsubject to 0 ≼ y ≼ 1y T 1 = n − k(434)This sequence is iterated until x T y vanishes; id est, until desired cardinality isachieved. This technique works often and, for some problem classes (beyondAx = b), it works all the time; meaning, optimal solution to problem (645)can often be found by this convex iteration. But examples can be found thatmake the iteration stall at a solution not of desired cardinality. Heuristicsfor breaking out of a stall can be implemented with some success:4.4.3 more cardinality and rank constraint examples4.4.3.0.1 Example. Sparsest solution to Ax = b.Given data, from Example 4.2.3.0.2,A =⎡−1 1 8 1 1 0⎢1 1 1⎣ −3 2 82 3−9 4 8144.26 e.g., branch and bound method.19− 1 2 31− 1 4 9⎤⎥⎦ , b =⎡⎢⎣11214⎤⎥⎦ (578)