v2010.10.26 - Convex Optimization

340 CHAPTER 4. SEMIDEFINITE PROGRAMMING4.5.1.5.1 Example. Sparsest solution to Ax = b. [70] [118](confer Example 4.5.1.8.1) Problem (682) has sparsest solution not easilyrecoverable by least 1-norm; id est, not by compressed sensing becauseof proximity to a theoretical lower bound on number of measurements mdepicted in Figure 100: for A∈ R m×nGiven data from Example 4.2.3.1.1, for m=3, n=6, k=1⎡⎤ ⎡ ⎤−1 1 8 1 1 01⎢1 1 1A = ⎣ −3 2 8 − 1 ⎥ ⎢2 3 2 3 ⎦ , b = ⎣−9 4 8141914 − 1 91214⎥⎦ (682)the sparsest solution to classical linear equation Ax = b is x = e 4 ∈ R 6(confer (695)).Although the sparsest solution is recoverable by inspection, we discernit instead by convex iteration; namely, by iterating problem sequence(156) (525) on page 334. From the numerical data given, cardinality ‖x‖ 0 = 1is expected. Iteration continues until x T y vanishes (to within some numericalprecision); id est, until desired cardinality is achieved. But this comes notwithout a stall.Stalling, whose occurrence is sensitive to initial conditions of convexiteration, is a consequence of finding a local minimum of a multimodalobjective 〈x, y〉 when regarded as simultaneously variable in x and y .(3.8.0.0.3) Stalls are simply detected as fixed points x of infeasiblecardinality, sometimes remedied by reinitializing direction vector y to arandom positive state.Bolstered by success in breaking out of a stall, we then apply convexiteration to 22,000 randomized problems:Given random data for m=3, n=6, k=1, in Matlab notationA=randn(3,6), index=round(5∗rand(1)) +1, b=rand(1)∗A(:,index)(769)the sparsest solution x∝e index is a scaled standard basis vector.Without convex iteration or a nonnegativity constraint x≽0, rate of failurefor this minimal cardinality problem Ax=b by 1-norm minimization of xis 22%. That failure rate drops to 6% with a nonnegativity constraint. If