sparse image representation via combined transforms - Convex ...

4.2. SPARSE DECOMPOSITION 83
interior point method. As we know, solving this LP problem in general takes more time
than doing a 2-D DCT or a 2-D wavelet transform.
Additional information about why we choose a minimum l 1 norm solution is in Section
4.2 and Section 4.3.
4.2 Sparse Decomposition
There is by now extensive research on finding sparse decompositions. These methods can
be roughly classified into three categories:
1. greedy algorithms,
2. global optimization algorithms,
3. special structures.
A global optimization algorithm searches for a decomposition minimizing a specified
objective function while satisfying some constraints. (Typically the objective function is
convex and any local minimizer is also a global minimizer.) The basis pursuit method (BP)
[25, 27] is a global optimization algorithm. In the noise free case, it minimizes ‖x‖ 1 subject
to Φx = y. Another example of the global optimization algorithm is the method of frames
(MOF) [34], which minimizes ‖x‖ 2 subject to Φx = y. Note that an ideal objective function
would be the l 0 norm of x, but that makes it a combinatorial optimization problem.
A greedy algorithm is a stepwise algorithm: at every step, the greedy algorithm takes one
or several elements out of the dictionary into a linear superposition of the desired image.
A well-known example is Matching Pursuit (MP) [102, 100]. The idea of MP is that at
every step, the algorithm picks the atom that is most correlated with the residual. Some
researchers give theoretical bounds for this greedy method, for example [112]. The newly
published high-resolution pursuit [89] is another example of greedy algorithms.
Some algorithms utilize the special structure of a dictionary. For example, best orthogonal
basis (BOB) [142] searches for an orthonormal basis that minimizes the additive entropy in
a dictionary that has a binary tree structure. The dictionary can be, for example, cosine
packets or wavelet packets.
Since a greedy algorithm is a stepwise algorithm, it runs the risk of being trapped in a
bad sequence. Some examples are given in [27, 40]. Some numerical experiments, together