sparse image representation via combined transforms - Convex ...

8 CHAPTER 2. SPARSITY IN IMAGE CODING
asymptotic result, because it based on an infinite amount of information. In practice,
both the source coder that compresses the message to the entropy rate and the channel
coder that transmits the bits at almost channel capacity may require intolerable amount
of computation and memory. In general, the use of a joint source-channel (JSC) code can
lead to the same performance but lower computational and storage requirements. Although
separate source and channel coding could be less effective, its appealing property is that a
separate source coder and channel coder are easy to design and implement.
From now on, we consider only source coding.
2.1.3 Transform Coding
The purpose of source encoding is to transfer a message into a bit stream; and source
decoding is essentially an inverse operation. There is no need to overstate the ubiquity
of transform coding in modern digital communication systems. For example, JPEG is an
industry standard for still image compression and transmission. It implements a linear
transform—a two-dimensional discrete cosine transform (2-D DCT). The MPEG standard
is an international standard for video compression and transmission that incorporates the
JPEG standards for still image compression. The MPEG standard could be the standard
for future digital television.
Figure 2.3 gives a general depiction of a modern digital communication system with embedded
transform coding. The input message is x. The operator T is a transform operator.
Most of the time, T is a linear time invariant (LTI) transform. After the transform, we
get coefficients y. Operator Q is a quantizer—it transfer continuous variables (in this case,
they are coefficients after transform T ) into discrete values. Without loss of generality, we
can assume each message is transferred into a bit stream of finite length. The operator E
is an entropy coder. It further shortens the bit stream by using, for example, run-length
coding for pictures [23], Huffman coding, arithmetic coding, or Lempel-Ziv coding, etc. Operators
T , Q and E together make the encoder. We assume that the bit stream is perfectly
transmitted through an error-corrected channel. The operators E −1 and Q −1 are inverse
operators of the entropy coder E and the quantizer Q, respectively. The operator U is the
reconstruction transform of transform T . If transform T is invertible, then the transform
U should be the inverse transform of T , or equivalently, U = T −1 . The final output at the
receiver is the estimation, denoted by ˆx, of the original message. Operators E −1 , Q −1 and
U together form the decoder.