In the Beginning was Information

A1 The Statistical View of InformationA1.1 Shannon’s Theory of InformationClaude E. Shannon (born 1916), in his well-known book “A mathematicaltheory of communications” [S7, 1948], was the first person to formulate amathematical definition of information. His measure of information, the“bit” (binary digit), had the advantage that quantitative properties ofstrings of symbols could be formulated. The disadvantage is just as plain:Shannon’s definition of information entails only one minor aspect of thenature of information as we will discuss at length. The only value of thisspecial aspect is for purposes of transmission and storage. The questionsof meaning, comprehensibility, correctness, worth or worthlessness arenot considered at all. The important questions about the origin (sender)and for whom it is intended (recipient) are also ignored. For Shannon’sconcept of information it is completely immaterial whether a sequence ofsymbols represents an extremely important and meaningful text, orwhether it was produced by a random process. It may sound paradoxical,but in this theory a random sequence of symbols represents the maximumvalue of information content – the corresponding value or number for ameaningful text of the same length is smaller.Shannon’s concept: His definition of information is based on a communicationsproblem, namely to determine the optimal transmission speed.For technical purposes the meaning and import of a message are of noconcern, so that these aspects were not considered. Shannon restrictedhimself to information that expressed something new, so that, briefly,information content = measure of newness, where “newness” does notrefer to a new idea, a new thought, or fresh news – which would haveencompassed an aspect of meaning. But it only concerns the surpriseeffect produced by a rarely occurring symbol. Shannon regards a messageas information only if it cannot be completely ascertained beforehand, sothat information is a measure of the unlikeliness of an event. An extremelyunlikely message is thus accorded a high information content. Thenews that a certain person out of two million participants has drawn thewinning ticket, is for him more “meaningful” than if every tenth personstood a chance, because the first event is much more improbable.Before a discrete source of symbols (N.B: not an information source!)delivers one symbol (Figure 31), there is a certain doubt as to which onesymbol a i of the available set of symbols (e. g. an alphabet with N lettersa 1 , a 2 , a 3 ,..., a N ) it will be. After it has been delivered, the previous uncertaintyis resolved. Shannon’s method can thus be formulated as the degree170