sparse image representation via combined transforms - Convex ...

4 CHAPTER 2. SPARSITY IN IMAGE CODING
fundamental problem of communication is that of reproducing at one point either exactly
or approximately a message selected at another point.” With these words, he described a
mathematical framework for communication systems that has become the most successful
model in communication and information theory.
Figure 2.1 gives Shannon’s illustration of a general communication system. (The figure
is reproduced according to the same figure in [127].) There are five components in this
system: information source, transmitter, channel, receiver, and destination.
1. An information source produces a message, or a sequence of messages, to be communicated
to the destination. A message can be a function of one or more continuous
or discrete variables and can itself be continuous- or discrete- valued. Some examples
of messages are: (a) a sequence of letters in telegraphy; (b) a continuous function
of time f(t) as in radio or telephony; (c) a set of functions r(x, y, t),g(x, y, t) and
b(x, y, t) having three variables—in color television they are the intensity of red, green
and blue as functions of spatial variables x and y and time variable t; (d) various
combinations—for example, television signals have both video and audio signals.
2. The transmitter, orencoder, transfers the message into a signal compatible with the
channel. In old-fashioned telephony, this operation consists of converting sound pressure
into a proportional electrical current. In telegraphy, we have an encoding system
to transfer a sequence of words to a sequence of dots, dashes and spaces. More complex
operations are applied to messages in modern communication systems.
3. The channel is the physical medium that conducts the transmitted signal. The mathematical
abstraction of the transmission is a perturbation by noise. A typical assumption
on the channel is that the noise is additive, but this assumption can be
changed.
4. The receiver, ordecoder, attempts to retrieve the message from the received signal.
Naturally, the decoding scheme depends on the encoding scheme.
5. The destination is the intended recipient of the message.
Shannon’s model is both concrete and flexible. It has been proven efficient for modeling
real world communication systems. From this model, some fundamental problems of
communication theory are apparent.