v2009.01.01 - Convex Optimization

326 CHAPTER 4. SEMIDEFINITE PROGRAMMING
processing and “root finding”) of a Dirac comb, the very same prototypical
signal from which Candès probabilistically derives minimum sampling
rate (Compressive Sampling and Frontiers in Signal Processing, University
of Minnesota, June 6, 2007). Combining their terminology, we paraphrase a
sparse sampling theorem:
Minimum sampling rate, asserted by Candès/Donoho, is proportional
to Vetterli’s rate of innovation (a.k.a, information rate, degrees of
freedom, ibidem June 5, 2007).
What distinguishes these researchers are their methods of reconstruction.
Properties of the 1-norm were also well understood by June 2004
finding applications in deconvolution of linear systems [74], penalized linear
regression (Lasso) [276], and basis pursuit [184]. But never before had
there been a formalized and rigorous sense that perfect reconstruction were
possible by convex optimization of 1-norm when information lost in a
subsampling process became nonrecoverable by classical methods. Donoho
named this discovery compressed sensing to describe a nonadaptive perfect
reconstruction method by means of linear programming. By the time Candès’
and Donoho’s landmark papers were finally published by IEEE in 2006,
compressed sensing was old news that had spawned intense research which
still persists; notably, from prominent members of the wavelet community.
Reconstruction of the Shepp-Logan phantom (Figure 86), from a severely
aliased image (Figure 88) obtained by Magnetic Resonance Imaging (MRI),
was the impetus driving Candès’ quest for a sparse sampling theorem. He
realized that line segments appearing in the aliased image were regions of high
total variation (TV). There is great motivation, in the medical community, to
apply compressed sensing to MRI because it translates to reduced scan-time
which brings great technological and physiological benefits. MRI is now
about 35 years old, beginning in 1973 with Nobel laureate Paul Lauterbur
from Stony Brook. There has been much progress in MRI and compressed
sensing since 2004, but there have also been two beacons pointing toward
abandonment of the 1-norm criterion (indigenous to reconstruction by
compressed sensing) in favor of a 0-norm: 4.39
4.39 Efficient techniques continually emerge that urge abandonment of the 1-norm criterion;
[71] e.g., five techniques for compressed sensing are compared in [33] demonstrating that
1-norm performance limits for cardinality minimization can be reliably exceeded.