sparse image representation via combined transforms - Convex ...

10 CHAPTER 2. SPARSITY IN IMAGE CODING
necessary to transmit a set of signals with prescribed fidelity can be reduced by transmitting
a set of linear combinations of the signals, instead of the original signals. Assuming
Gaussian signals and RMS or mean-squared error (MSE) distortion, the Karhunen-Loéve
transform (KLT) is optimal. Later, Huang and Schultheiss [84, 85] extended this idea to
a system that includes a quantizer. Nowadays, PCA has become a ubiquitous technique
in multivariate analysis. Its equivalent—KLT—has become a well-known method in the
electrical engineering community.
To find a transform that will generate even sparser coefficients than most of the existing
transform is the ultimate goal of this thesis. Our key idea is to combine several state-of-theart
transforms. The combination will introduce redundancy, but at the same time it will
provide flexibility and potentially lead to a decomposition that is sparser than one with a
single transform.
When describing image coding, we cannot avoid mentioning three topics: quantization,
entropy coding and predictive coding. Each of the following subsubsections is dedicated to
one of these topics.
Quantization
All quantization schemes can be divided into two categories: scalar quantization and vector
quantization. We can think of vector quantization as an extension of scalar quantization in
high-dimensional space. Gray and Neuhoff 1998 [76] provides a good review paper about
quantization. Two well-written textbooks are Gersho and Gray 1992 [70] and Sayood 1996
[126].
The history of quantization started from Pulse-code modulation (PCM), which was
patented in 1938 by Reeves [122]. PCM was the first digital technique for conveying an
analog signal over an analog channel. Twenty-five years later, a noteworthy article [37] was
written by Reeves and Deloraine. It gave a historical review and an appraisal of the future
of PCM. Some predictions in this article turned out to be prophetic, especially the one
about the booming of digital technology.
Here we give a brief description of scalar quantization. A scalar quantization is a mapping
from a real-valued source to a discrete subset; symbolically,
Q : R → C,