sparse image representation via combined transforms - Convex ...

18 CHAPTER 2. SPARSITY IN IMAGE CODING
l p space if and only if θ has a finite weak l p norm. For fixed p, the statement “Θ ⊂ wl p ”
simply means that every sequence in Θ has a finite weak l p norm. The critical index of a
functional space Θ, denoted by p ∗ (Θ), is defined as the infimum of p such that the weak l p
space includes Θ:
p ∗ (Θ) = inf{p :Θ⊂ wl p }.
In Section 2.2.2, we describe the linkage between critical index and optimal exponent,
where the latter is a measure of efficiency of “optimal” coding in a certain functional space.
A Result in the Finite Case
Before we move into the asymptotic discussion, the following result provides insight into
coding a finite-length sequence. The key idea is that the faster the finite sequence decays,
the fewer bits required to code the sequence. Here the sparsity is measured by the decay of
the sorted amplitudes, which in the 1-D case is the decay of the sorted absolute values of a
finite-length sequence.
In general, it is hard to define a “universal” measure of speed of decay for a finite-length
sequence. We consider a special case. The idea is depicted in Figure 2.5. Suppose that
two curves of sorted amplitudes of two sequences (indicated by A and B in the figure) have
only one intersection. We consider the curve that is lower at the tail part (indicated by
B in the figure) corresponds to a sparser sequence. (Obviously, this is a simplified version
because there could be more than one intersection.) We prove that by using the most
straightforward coding scheme, which is to take an identical uniform scalar quantizer at
every coordinate, we need fewer bits to code B than to code A.
To be more specific, without loss of generality, suppose we have two normalized nonincreasing
sequences. Both of these sequences have L 2 norm equal to 1. One decays “faster”
than the other, in the sense that when after a certain index, the former sorted-amplitude
sequence is always below the latter one, as in Figure 2.5. (Curve A is always above curve
B after the intersection.) Suppose we deploy a uniform scalar quantizer with the same
quantization parameter q on every coordinate. Then the number of bits required to code
the ith element θ i in the sequence θ is no more than log 2 [θ i /q]+1(where[x] is the closest
integral value to x). Hence the total number of bits to code vector θ = {θ i :1≤ i ≤ N}
is upper bounded by ∑ N
i=1 log 2[θ i /q] +N. Since q and N are constants, we only need to