In the Beginning was Information

as expressing the highest possible freedom from interference. In this waya very exciting basic phenomenon of physiology could be understood bymeans of the new concepts of information theory.”It should now be clear that Shannon’s information theory is very importantfor evaluating transmission processes of messages. But, as far as the messageitself is concerned, it can only say something about its statisticalproperties, and nothing about the essential nature of information. This isits real weakness as well as its inherent propensity for leading to misunderstandings.The German cyberneticist Bernhard Hassenstein rightlycriticises it in the following words: “It would have been better to devisean artificial term, rather than taking a common word and giving it a completelynew meaning.” If we restrict Shannon’s information to one of thefive aspects of information, then we do obtain a scientifically sound solution[G5]. But without the extension to the other four levels of information,we are stuck with the properties of a transmission channel. No science,apart from communication technology, should limit itself to just thestatistical level of information.Natural languages may be analysed and compared statistically by meansof Shannon’s theory, as we will now proceed to do.A1.4 Statistical Analysis of LanguageIt is possible to calculate certain quantitative characteristics of languagesby means of Shannon’s information theory. One example of such a propertyis the average information content of a letter, a syllable, or a word. Inequation (9) this numerical value is denoted by H, the entropy.1. Letters: If, for the sake of simplicity, we assume that all 26 letters plusthe space between words occur with the same frequency, then we haveH 0 = lb 27 = log 27/log 2 = 4.755 bits/letter. (11)But it is known that the frequency of occurrence of the different letters ischaracteristic of the language we are investigating [B2 p 4]. The probabilityp i of occurrence of single letters and the space are given for Englishand German in Table 1, as well as the average information content per letter,H. On applying equation (9) to the various letter frequencies p i in German,the average information content (= entropy) of a symbol is given by19630H 1 = ∑ p i x lb(1/p i ) = 4.112 95 bits/letter. (12)i=1