082-Engineering-Mathematics-Anthony-Croft-Robert-Davison-Martin-Hargreaves-James-Flint-Edisi-5-2017

918 Chapter 28 Probability
Engineeringapplication28.10
Redundancyofabinarydatastream
Consider the source of binary digits examined in Engineering application 28.6 and
28.9. The maximum entropy for a binary stream is 1 bit per binary digit. Calculate
the redundancy ineach case.
Solution
For Engineering application 28.6
redundancy = 1 −1 = 0
1
For Engineering application 28.9
redundancy = 1 −0.881
1
= 0.119
Engineeringapplication28.11
Redundancyofacharacterdatastream
A stream of data consists of four characters A, B, C, D with probabilities 0.1, 0.3,
0.2,0.4,respectively. Calculate the redundancy.
Solution
It can be shown that the maximum entropy, H max
, corresponds to the situation in
whichthe probabilityofeachsymbolisthe same, thatis0.25.
H max
= 4 × (−0.25log 2
(0.25)) = 2 bits
Theactual entropy,H act
, isgiven by
H act
= −(0.1log 2
(0.1) +0.3log 2
(0.3) +0.2log 2
(0.2) +0.4log 2
(0.4))
= 1.846bits
redundancy = 2 −1.846
2
= 0.0770
Intheexampleswehaveexaminedsofarwehaveusedthebitastheunitofinformation
because the most common form of digital signalling uses binary digits. When there are
only two possible events it is possible to represent an event by a single binary digit.
However, if there is a larger number of possible events then several binary digits are
neededtorepresentasingleevent.Whencalculatingvaluesforinformationandentropy
in these examples an assumption was made that each event was represented by binary
sequences or codes of the same length. It is only possible to do this efficiently if the
numberofeventsisapowerof2,thatis2,4,8,16, ....Inpracticethisproblemdoesnot
arise because it is more common to produce codes that have a small number of binary
digitsforlikelyeventsandalongnumberofbinarydigitsforunlikelyevents.Thisallows