sparse image representation via combined transforms - Convex ...

4.3. MINIMUM l 1 NORM SOLUTION 85
l 0 norm is a nonconvex function while l 1 norm is convex.
Some inspiring applications of the minimum l 1 norm phenomenon are given in [55]:
• Recovering missing segments of a bandlimited signal. Suppose s is a bandlimited
signal with frequency components limited in B. Buts is missing in a time interval T .
Given B, if|T | is small enough, then the minimizer of ‖s − s ′ ‖ 1 among all s ′ whose
frequency components are limited in B is exactly s.
• Recovery of a “sparse” wide-band signal from narrow-band measurements. Suppose s
is a signal that is “sparse” in the time domain. As we know, a time-sparse signal must
occupy a wide band in the frequency domain. Suppose N t ≥|s|. Suppose we can only
observe a bandlimited fraction of s, r = P B s, here P B functioning like a bandpass
filter. We can perfectly recover s by finding the minimizer of ‖r−P B s ′ ‖ 1 among all the
s ′ satisfying |s ′ |≤N t . This phenomenon has application in several branches of applied
science, where instrumental limitations make the available observation bandlimited.
On the other hand, if we consider the symmetry between the time domain and the
frequency domain, this is a dual of the previous case. We simply switch the positions
of the time domain and the frequency domain.
Another recent advance in theory [43] has given an interesting result. Suppose that the
overcomplete dictionary we consider is a combination of two complementary orthonormal
bases. By “complementary” we mean that the maximum absolute value of the inner product
of any two elements (one from each basis) is upper bounded by a small value. Suppose the
observed signal y is made by a small number of atoms in the dictionary. Solving the
minimum l 1 norm problem will give us the same solution as solving the minimum l 0 norm
problem. More specifically, if a dictionary is made by two bases—Dirac basis and Fourier
basis—and if the observation y is made by fewer than √ N/2 atoms from the dictionary,
where N is the size of the signal, then the minimum l 1 norm decomposition is the same as
the minimum l 0 norm decomposition.
‖x‖ 0 denotes the number of nonzero elements in the vector x. Thel 0 norm is generally
regarded as the measure of sparsity. So the minimum l 0 norm decomposition is generally
regarded as the sparsest decomposition. Note the minimum l 0 norm problem is a combinatorial
problem and in general is NP hard. But the minimum l 1 norm problem is a convex
optimization problem and can be solved by some polynomial-time optimization methods,