院試情報理論解答例

11 2005 7 13 17 1H(Y ) = H(q)= −q log 2 q − (1 − q) log 2 (1 − q)0.250.50.75 2 1 0.25 0.5 0.75 1 0 1 p 12 1 1 p 22q = (1 − p)p 12 + pp 22= 0.1(1 − p) + 0.6p= 0.1 + 0.5p0.250.50.75p 0 1 q 0.1 0.6 0.250.5 0.75 1 3p = 2q − 0.21

0.250.50.7510.25 0.75 1 0 I(X;Y) 0.5§H(Y|X) ¦0.5¦15¦1H(Y |X) 2∑2∑H(Y |X) = (1 − p) p 1i − log 2 p 1i + p p 2i − log 2 p 2ii=1i=1= (1 − p)H(p 11 ) + pH(p 22 )= (1 − p)H(0.9) + pH(0.6)= (1.2 − 2q)H(0.9) + (2q − 0.2)H(0.6)= 2 {H(0.6) − H(0.9)} q + 1.2H(0.9) − 0.2H(0.6)(H(0.6) > H(0.9) )I(X; Y ) = H(Y ) − H(Y |X)= H(q) − H(Y |X) 1: H(Y |X)1 0.75 0.5 0.25 2: I(X; Y ) 4I(X; Y ) q = 0.35 C C = max(I(X; Y )) (p )p I(X; Y ) 0.1 0.6 q=0.35 2

0.750.50.250.25 0.5 0.75 1 0¦0.25¦05 5q = (1 − p)(1 − p 11 ) + 0.6pp = q + p 11 − 1p 11 − 0.41 − p = 0.6 − qp 11 − 0.4 p I(X; Y ) I(X; Y ) = H(Y ) − H(Y |X)= H(q) − {(1 − p)H(p 11 ) + p(0.6)}{ 0.6 − q= H(q) −p 11 − 0.4 H(p 11) + q + p }11 − 1p 11 − 0.4 H(0.6) I(X; Y ) q (1 − p 11 ) 0.6 q = (1 − p 11) + 0.62= 1.6 − p 112 I(X; Y ) C I(X; Y ) q = 1.6−p 112( ) { 1.6 − p11 1I(X; Y ) = H−2 2 H(p 11) + 1 }2 H(0.6)q p = 1 − p I(X; Y ) 3

1100 16 1H(S) = H 1 (S) = −0.9 log 2 0.9 − 0.1 log 2 0.1= −0.9 {log 2 9 − log 2 10} − 0.1 {log 2 1 − log 2 10}= −0.9(3.17 − 3.32) − 0.1(0 − 3.32)= −0.9 × (−0.15) − 0.1 × (−3.32)= 0.135 + 0.332= 0.467 22 a 0,b 1 L 1 = 0.9 × 1 + 0.1 × 1 = L 1 > H(S) 3 2 L 2 1 2L 22L 2 = 0.81 × 1 + 0.09 × 2 + 0.09 × 3 + 0.01 × 3= 0.81 + 0.18 + 0.27 + 0.03= 1.29 L 2 = 0.645 L 1 >aa(0.81)→0L 2 >H(S)ab(0.09)→10 ba(0.09)→110 bb(0.01)→11110 4n S n M p 1 , . . . , p M l 1 , . . . , l M 2 −l 1+ · · · + 2 −l M≤ 14

q 1 + · · · + q M ≤ 1 q i M∑M∑− p i log 2 p i ≤ − p i log 2 q ii=1i=1q i = 2 −li q 1 + · · · +q M ≤ 1 M∑H(S n ) = − p i log 2 p ii=1M∑≤ − p i log 2 q i= −i=1M∑p i log 2 2 −l ii=1M∑= p i l ii=1= nL n− log 2 p i ≤ l i < − log 2 p i + 1 (1) l i l i 2 −li ≤ 2 log 2 pi = p iM∑M∑2 −l i≤ p i = 1i=1i=1 {l i } 1 p i −M∑p i log 2 p i ≤i=1∑ Mi=1 p il i < −M∑M∑p i log 2 p i +i=1H(S n ) ≤ nL n < H(S n ) + 1i=1p i H(S n ) ≤ nL n < H(S n ) + 1 5

ab(0.09)→010 501 1001H(S) = lim H H(S n )n(S) = limn→∞ n→∞ nb(0.1)→00 6S 1 aaa(0.729)→1 aab(0.081)→0110.1 × 2 + 0.09 × 3 + 0.081 × 3 + 0.729 × 1= 0.2 + 0.27 + 0.243 + 0.729= 1.4420.1 × 1 + 0.09 × 2 + 0.081 × 3 + 0.729 × 3= 0.l + 0.18 + 0.243 + 2.187= 2.71 1 L r = 1.4422.71 0.532 L 1 > L 2 > L r > H(S)6