sparse image representation via combined transforms - Convex ...

72 CHAPTER 3. IMAGE TRANSFORMS AND IMAGE FEATURES
The idea of the BOB is the following: if the entropy associated with the two subintervals is
lower than the entropy associated with the larger dyadic interval, then we choose to divide
the larger dyadic interval. This process is repeated until there is no more partitioning to
do. The final partition of the unit interval [0, 1) determines a subset of localized cosine
functions. It’s not hard to observe that this subset of local cosine functions makes an
orthonormal basis. For a given signal, instead of computing the entropy of a statistical
distribution, we can compute the empirical entropy, which is just the entropy of a set of
coefficients. Note that the coefficients are from a transform of the signal. Based on this
empirical entropy, we can use the BOB algorithm to choose a subset of coefficients, and
correspondingly a subset of basis functions that form a basis.
Complexity. Suppose the length of the signal is N, forN ∈ N. Note that when we take
cosine functions as an orthonormal basis over an interval, the “folding” transform gives
another orthonormal system whose elements are functions that are defined on the entire
real axis, but are localized (have finite support). For every dyadic interval, we have such
an orthonormal system. Combining all these systems, we actually get the cosine packets
(CP). Suppose dyadic intervals having the same length is a cover of the entire real axis. We
say that CP elements associated with these intervals are at the same scale. For length-N
signals, we have log 2 N scales. To calculate the CP coefficients at one scale, we can first
apply “folding” to the data, then apply a DCT. The folding is an O(N) operation. The
discrete cosine transform has O(N log N) complexity. Since we have log 2 N scales, the total
complexity of computing CP coefficients is O(N log 2 N).
Wavelet Packets
In MRA, instead of dividing the low-frequency part, or a scaling space (a space spanned
by scaling functions), we can apply a quadratic mirror filter to divide the high-frequency
part, or a wavelet space (which is spanned by the wavelet functions). This idea unveils
developments of wavelet packets. Equivalently, in the filter banks algorithm, at each step,
instead of deploying a pair of filters after an LPF, we can deploy a pair of filters after an
HPF. By doing this, we partition the high-frequency part. Since a wavelet space contains
high-frequency components, partition in a high-frequency part is equivalent to partition in
a wavelet space.
In every step, we can adaptively choose to deploy a pair of filters either after an LPF
or an HPF. In this framework, all possible sets of coefficients have a binary tree structure.