v2010.10.26 - Convex Optimization

4.5. CONSTRAINING CARDINALITY 3474.5.1.6 Compressed sensing geometry with a nonnegative variableIt is well known that cardinality problem (530), on page 233, is easier to solveby linear programming when variable x is nonnegatively constrained thanwhen not; e.g., Figure 71, Figure 100. We postulate a simple geometricalexplanation: Figure 70 illustrates 1-norm ball B 1 in R 3 and affine subsetA = {x |Ax=b}. Prototypical compressed sensing problem, for A∈ R m×nminimize ‖x‖ 1xsubject to Ax = b(518)is solved when the 1-norm ball B 1 kisses affine subset A .If variable x is constrained to the nonnegative orthantminimize ‖x‖ 1xsubject to Ax = bx ≽ 0(523)then 1-norm ball B 1 (Figure 70) becomes simplex S in Figure 103.Whereas 1-norm ball B 1 has only six vertices in R 3 corresponding tocardinality-1 solutions, simplex S has three edges (along the Cartesian axes)containing an infinity of cardinality-1 solutions. And whereas B 1 has twelveedges containing cardinality-2 solutions, S has three (out of total four)facets constituting cardinality-2 solutions. In other words, likelihood of alow-cardinality solution is higher by kissing nonnegative simplex S (523) thanby kissing 1-norm ball B 1 (518) because facial dimension (corresponding togiven cardinality) is higher in S .Empirically, this conclusion also holds in other Euclidean dimensions(Figure 71, Figure 100).4.5.1.7 cardinality-1 compressed sensing problem always solvableIn the special case of cardinality-1 solution to prototypical compressedsensing problem (523), there is a geometrical interpretation that leadsto an algorithm more efficient than convex iteration. Figure 103 showsa vertex-solution to problem (523) when desired cardinality is 1. Butfirst-quadrant S of 1-norm ball B 1 does not kiss line A ; which requiresexplanation: Under the assumption that nonnegative cardinality-1 solutionsexist in feasible set A , it so happens,