sparse image representation via combined transforms - Convex ...

20 CHAPTER 2. SPARSITY IN IMAGE CODING
exponent. We first introduce the definition of optimal exponent, then cite the result that
states the connection between critical index and optimal exponent.
Optimal Exponent
We consider a functional space Θ made by infinite-length real-valued sequences. An ɛ-ball
in Θ, which is centered at θ 0 , is (by definition) B ɛ (θ 0 )={θ : ‖θ − θ 0 ‖ 2 ≤ ɛ, θ ∈ Θ}, where
‖·‖ 2 is the l 2 norm in Θ and θ 0 ∈ Θ. An ɛ-net of Θ is a subset of Θ {f i : i ∈ I,f i ∈ Θ} that
satisfies Θ ⊂ ⋃ i∈I B ɛ(f i ), where I is a given set of indices. Let N(ɛ, Θ) denote the minimum
possible cardinality of the ɛ-net {f i }. The (Kolmogorov-Tikhomirov) ɛ-entropy of Θ is (by
definition)
H ɛ (Θ) = log 2 N(ɛ, Θ).
The ɛ-entropy of Θ, H ɛ (Θ), is to within one bit the minimum number of bits required
to code in space Θ with distortion less than ɛ. If we apply the nearest-neighbor coding,
the total number of possible cases needed to record is N(ɛ, Θ), which should take no more
than ⌈log 2 N(ɛ, Θ)⌉ bits. (The value ⌈x⌉ is the smallest integer that is no smaller than x.)
Obviously the ɛ-entropy of Θ, which is denoted by H ɛ (Θ), is within 1 bit of the total number
of bits required to code in functional space Θ.
It is not hard to observe that when ɛ → 0, H ɛ (Θ) → +∞. Now the question is how fast
the ɛ-entropy H ɛ (Θ) increases when ɛ goes to zero. The optimal exponent defined in [47] is
a measure of the speed of increment:
α ∗ (Θ) = sup{α : H ɛ (Θ) = O(ɛ −1/α ),ɛ→ 0}.
We refer readers to the original paper [47] for a more detailed discussion.
Asymptotic Result
The key idea is that there is a direct link between the critical index p ∗ and the optimal
exponent α ∗ . The following is cited from [47], Theorem 2.
Theorem 2.2 Let Θ be a bounded subset of l 2 that is solid, orthosymmetric, and minimally