In the Beginning was Information

As explained fully in Appendix A1, Shannon’s theory of informationis suitable for describing the statistical aspects of information,e. g. those quantitative properties of languages which dependon frequencies. Nothing can be said about the meaningfulness ornot of any given sequence of symbols. The question of grammaticalcorrectness is also completely excluded at this level. Conclusions:Definition 1: According to Shannon’s theory any randomsequence of symbols is regarded as information, without regardto its origin, nor whether it is meaningful or not.Definition 2: The statistical information content of a sequenceof symbols is a quantitative concept, measured in bits (binarydigits).According to Shannon’s definition the information content of a singlemessage (which could be one symbol, one sign, one syllable, ora single word) is a measure of the probability of its being receivedcorrectly. Probabilities range from 0 to 1, so that this measure isalways positive. The information content of a number of messages(signs for example) is found by adding the individual probabilitiesas required by the condition of summability. An important propertyof information according to Shannon is:Theorem 4: A message which has been subject to interferenceor “noise”, in general comprises more information than anerror-free message, according to Shannon’s theory.This theorem follows from the larger number of possible alternativesin a distorted message, and Shannon states that the informationcontent of a message increases with the number of symbols(see equation 6 in Appendix A1). It is obvious that the actual informationcontent cannot at all be described in such terms, as shouldbe clear from the following example: When somebody uses manywords to say practically nothing, this message is accorded a largeinformation content because of the large number of letters used.But if somebody else, who is really knowledgeable, conciselyexpresses the essentials, his message has a much lower informationcontent.55