v2010.10.26 - Convex Optimization

372 CHAPTER 4. SEMIDEFINITE PROGRAMMINGMinimum sampling rate:of Ω-bandlimited signal: 2Ω (Shannon)of k-sparse length-n signal: k log 2 (1+n/k)(Candès/Donoho)(Figure 100). Certainly, much was already known about nonuniformor random sampling [36] [208] and about subsampling or multiratesystems [91] [357]. Vetterli, Marziliano, & Blu [366] had congealed atheory of noiseless signal reconstruction, in May 2001, from samples thatviolate the Shannon rate. They anticipated the sparsifying transformby recognizing: it is the innovation (onset) of functions constituting a(not necessarily bandlimited) signal that determines minimum samplingrate for perfect reconstruction. Average onset (sparsity) Vetterli et aliicall rate of innovation. The vector inner-products that Candès/Donohocall samples or measurements, Vetterli calls projections. From thoseprojections Vetterli demonstrates reconstruction (by digital signal processingand “root finding”) of a Dirac comb, the very same prototypicalsignal from which Candès probabilistically derives minimum samplingrate (Compressive Sampling and Frontiers in Signal Processing, Universityof Minnesota, June 6, 2007). Combining their terminology, we paraphrase asparse sampling theorem:Minimum sampling rate, asserted by Candès/Donoho, is proportionalto Vetterli’s rate of innovation (a.k.a, information rate, degrees offreedom, ibidem June 5, 2007).What distinguishes these researchers are their methods of reconstruction.Properties of the 1-norm were also well understood by June 2004finding applications in deconvolution of linear systems [83], penalized linearregression (Lasso) [318], and basis pursuit [213]. But never before hadthere been a formalized and rigorous sense that perfect reconstruction werepossible by convex optimization of 1-norm when information lost in asubsampling process became nonrecoverable by classical methods. Donohonamed this discovery compressed sensing to describe a nonadaptive perfectreconstruction method by means of linear programming. By the time Candès’and Donoho’s landmark papers were finally published by IEEE in 2006,compressed sensing was old news that had spawned intense research whichstill persists; notably, from prominent members of the wavelet community.