sparse image representation via combined transforms - Convex ...

84 CHAPTER 4. COMBINED IMAGE REPRESENTATION
with some recent advances in theoretical study, show that a global optimization algorithm
like BP is more stable in recovering the original sparse decomposition, if it exists. But BP
is a computationally intensive method. The remainder of this thesis is mainly devoted to
overcoming this barrier. In the next section, Section 4.3, we first explain the optimality of
the minimum l 1 norm decomposition and then give our formulation, which is a variation
of the exact minimum l 1 norm decomposition. This formulation determines the numerical
problem that we try to solve.
4.3 Minimum l 1 Norm Solution
The minimum l 1 norm solution means that in a decomposition y =Φx, where y is the desired
signal/image, Φ is a flat matrix with each column being an atom from an overcomplete
dictionary and x is a coefficient vector, we pick the one that has the minimum l 1 norm
(‖x‖ 1 ) of the coefficient vector x.
A heuristic argument about the optimality of the minimum l 1 norm decomposition is
that it is the “best” convexification of the minimum l 0 norm problem. Why? Suppose we
consider all the convex functions that are supported in [−1, 1] and upper bounded by the
l 0 norm function and we solve
⎧
⎪⎨ f(x) ≤‖x‖ 0 ,
maximize f(x), subject to ‖x‖ ∞ ≤ 1,
x
⎪⎩
f is convex.
The solution of the above problem is ‖x‖ 1 .
The ideas of using the minimum l 1 norm solutions in signal estimation and recovery date
back to 1965, in which Logan [96] described some so-called minimum l 1 norm phenomena.
Another good reference on this topic is [55]. The idea of the minimum l 1 norm phenomenon
is the following: a signal cannot be “sparse” in both time and Fourier (frequency) domain;
if we know the signal is limited in one domain (e.g., frequency domain) and the signal is
unknown in a relatively small set in another domain (e.g., time domain), then we may be
able to perfectly recover the signal by solving the minimum l 1 norm problem.
This phenomenon is connected to the uncertainty principle. But the conventional uncertainty
principle is based on the l 0 norm. In the continuous case, the l 0 norm is the length
of interval; in the discrete case, the l 0 norm is the cardinality of a finite set. As we know,