v2009.01.01 - Convex Optimization

4.5. CONSTRAINING CARDINALITY 305
Whereas 1-norm ball B 1 has only six vertices in R 3 corresponding to
cardinality-1 solutions, simplex S has three edges (along the Cartesian
axes) containing an infinity of cardinality-1 solutions. And whereas B 1
has twelve edges containing cardinality-2 solutions, S has three (out of
total four) two-dimensional faces constituting cardinality-2 solutions. In
other words, likelihood of a low-cardinality solution is higher by kissing
nonnegative simplex S (691) than by kissing 1-norm ball B 1 (460) because
facial dimension (corresponding to given cardinality) is higher in S .
Empirically, this conclusion also holds in other Euclidean dimensions
(Figure 62, Figure 80).
4.5.1.4 cardinality-1 compressed sensing problem always solvable
In the special case of cardinality-1 solution to prototypical compressed
sensing problem (691), there is a geometrical interpretation that leads
to an algorithm more efficient than convex iteration. Figure 83 shows
a vertex-solution to problem (691) when desired cardinality is 1. But
first-quadrant S of 1-norm ball B 1 does not kiss line A ; which requires
explanation: Under the assumption that nonnegative cardinality-1 solutions
exist in feasible set A , it so happens,
columns of measurement matrix A , corresponding to any particular
solution (to (691)) of cardinality greater than 1, may be deprecated and
the problem solved again with those columns missing. Such columns
are recursively removed from A until a cardinality-1 solution is found.
Either a solution to problem (691) is cardinality-1 or column indices of A ,
corresponding to a higher cardinality solution, do not intersect that index
corresponding to the cardinality-1 solution. When problem (691) is first
solved, in the example of Figure 83, solution is cardinality-2 at the kissing
point • on the indicated edge of simplex S . Imagining that the corresponding
cardinality-2 face F has been removed, then the simplex collapses to a line
segment along one of the Cartesian axes. When that line segment kisses
line A , then the cardinality-1 vertex-solution illustrated has been found. A
similar argument holds for any orientation of line A and point of entry to
simplex S .
Because signed compressed sensing problem (460) can be equivalently
expressed in a nonnegative variable, as we learned in Example 3.1.3.0.1
(p.203), and because a cardinality-1 constraint in (460) transforms to