sparse image representation via combined transforms - Convex ...

16 CHAPTER 2. SPARSITY IN IMAGE CODING
First, the formulation. Suppose θ = {θ i : i ∈ N} is an infinite-length real-valued
sequence: θ ∈ R ∞ . Further assume that the sequence θ can be sorted by absolute values.
Suppose the sorted sequence is {|θ| (i) ,i∈ N}, where |θ| (i) is the i-th largest absolute value.
The weak l p norm of the sequence θ is defined as
|θ| wl p =supi 1/p |θ| (i) . (2.3)
i
Note that this is a quasi-norm. (A norm satisfies a triangular inequality, ‖x+y‖ ≤‖x‖+‖y‖;
a quasi-norm satisfies a quasi-triangular inequality, ‖x+y‖ ≤K(‖x‖+‖y‖) for some K>1.)
The weak l p norm has a close connection with three other measures that are related to vector
sparsity.
Numerosity
A straightforward way to measure the sparsity of a vector is via its numerosity: the number
of elements whose amplitudes are above a given threshold δ. In a more mathematical
language, for a fixed real value δ, the numerosity is equal to #{i : |θ i | >δ}. The following
lemma is cited from [47].
Lemma 2.1 For any sequence θ, the following inequality is true:
#{i : |θ i | >δ}≤|θ| p wl p δ −p , δ > 0.
From the above lemma, a small weak l p norm leads to a small number of elements that
are significantly above zero. Since numerosity, which basically counts significantly large
elements, is an obvious way to measure the sparsity of a vector, the weak l p norm is a
measure of sparsity.
Compression Number
Another way to measure sparsity is to use the compression number, defined as
( ∞ 1/2
∑
c(n) = |θ|
(i)) 2 .
i=n+1
Again, the compression number is based on the sorted amplitudes. In an orthogonal basis,
we have isometry. If we perform a thresholding scheme by keeping the coefficients associated