In the Beginning was Information

A1.2 Mathematical Description of StatisticalInformationA1.2.1 The Bit: Statistical Unit of InformationOne of the chief concerns in science and technology is to express resultsas far as possible in a numerical form or in a formula. Quantitative measuresplay an important part in these endeavours. They comprise twoparts: the relevant number or magnitude, and the unit of measure. The latteris a pre-determined unit of comparison (e. g. metre, second, Watt)which can be used to express other similarly measurable quantities.The bit (abbreviated from binary digit) is the unit for measuring informationcontent. The number of bits is the same as the number of binary symbols.In data processing systems information is represented and processedin the form of electrical, optical or mechanical signals. For this purpose itis technically extremely advantageous, and therefore customary, toemploy only two defined (binary) states and signals. Binary states havethe property that only one of the two binary symbols can be involved at acertain moment. One state is designated as binary one (1), and the other asbinary nought (0). It is also possible to have different pairs of binary symbolslike 0 and L, YES and NO, TRUE and FALSE, and 12 V and 2 V. Incomputer technology a bit also refers to the binary position in a machineword. The bit is also the smallest unit of information that can be representedin a digital computer.When text is entered in a computer, it is transformedinto a pre-determined binary code and also stored in this form.One letter usually requires 8 binary storage positions, known as a byte.The information content (= storage requirement) of a text is thendescribed in terms of the number of bits required. Different pieces of textare thus accorded the same information content, regardless of sense andmeaning. The number of bits only measures the statistical quantity of theinformation, with no regard to meaningfulness.Two computer examples will now illustrate the advantages (e. g. to helpdetermine the amount of storage space) and the disadvantages (e. g. ignoringthe semantic aspects) of Shannon’s definition of information:Example 1: Storage of biological information: The human DNA molecule(body cell) is about two metres long when fully stretched and it containsapproximately 6 x 10 9 nucleotides (the chemical letters: adenin,cytosin, guanin, and thymin). How much statistical information is thisaccording to Shannon’s defintion? The N = 4 chemical letters, A, C, G,and T occur nearly equally frequently; their mean information content isH = lb 4 = (log 4)/(log 2) = 2 bits. The entire DNA thus has an informationcontent of I tot = 6 x 10 9 nucleotides x 2 bits/nucleotide = 1.2 x 10 10 bits178