In the Beginning was Information

) Expectation value of the information content of a symbol: H can alsobe regarded as the expectation value of the information content of a symbolarriving from a continuously transmitting source.c) The mean decision content per symbol: H can also be regarded as themean decision content of a symbol. It is always possible to encode thesymbols transmitted by a source of messages into a sequence of binarysymbols (0 and 1). If we regard the binary code of one symbol as a binaryword, then H can also be interpreted as follows (note that binary words donot necessarily have the same length): It is the average word length of thecode required for the source of the messages. If, for instance, we wantto encode the four letters of the genetic code for a computer investigationand the storage requirements have to be minimised, then H will be lb 4 = 2binary positions (e. g. 00 = A, 01 = C, 10 = G, and 11 = T).The exceptional case of symbols having equal probabilities: This is animportant case, namely that all N symbols of the alphabet or some otherset of elements occur with the same probability p(x i ) = 1/N. To find themean information content of a single symbol, we have to divide the rightside of equation (8) by n:H ≡ I ave (x) ≡ i = lb N (10)We now formulate this statement as a special theorem:Theorem A1: In the case of symbol sequences of equal probability(e. g. the digits generated by a random number generator) the averageinformation content of a symbol is equal to the information content ofeach and every individual symbol.176