Stochastic Processes II (FP-7.5) Solution Set 6

Stochastic Processes II (FP-7.5)Solution Set 6Problem 6.1 (Problem 7.15 in Shanmugan)Solution:• Orthogonality principleE [( Y (n) − Ŷ (n)) X(n − k) ] = 0, ∀k = −M, ..., +M (1)Substituting (1) from the exercise set in the left-hand of equation (1) above andapplying the property of WSS process yieldsE[Y (n)X(n − k)] −} {{ }=E[X(n)Y (n+k)]=R XY (k)+M∑m=−Mh(m) E[X(n − k)X(n − m)]} {{ }=E[X(n)X(n+k−m)]=R XX (k−m)= 0HenceR XY (k) =+M∑m=−Mh(m)R XX (k − m), k = −M, ..., +MWe can rewrite the 2M + 1 equations above in a matrix form as⎡⎤ ⎡⎤R XY (−M)R XX (0) R XX (1) . . . R XX (2M − 1) R XX (2M)R XY (−M + 1)R XX (1) R XX (0) . . . R XX (2M − 2) R XX (2M − 1)⎢.⎥ ⎢....⎥⎣ R XY (M − 1) ⎦ ⎣R XX (2M − 1) R XX (2M − 2) . . . R XX (0) R XX (1) ⎦R XY (M)R XX (2M) R XX (2M + 1) . . . R XX (1) R XX (0)} {{ } } {{ }R XY=⎡ ⎤h(−M)h(−M + 1)·⎢.⎥⎣ h(M − 1) ⎦}h(M){{ }h• Coefficient vector of the LMMSEE:Provided R XX is invertible,h = R −1XX R XY .Such a LMMSEE is also called a finite Wiener filter.R XX(2)1

Problem 6.2Solution:a) Find the Wiener-Hopf equations for the coefficients of the noncausal Wiener filterof length 2M + 1 and the causal Wiener filter of length M + 1 estimating Y (n)based on the observation of X(n).- Non-causal finite Wiener filter with finite length 2M + 1:Ŷ (n) =+M∑m=−Mh(m)X(n − m).We can make use of the result in Problem 7.15 to find the coefficients of thenoncausal Wiener filter:R XY (k) = E[X(n)Y (n + k)]= E[(Y (n) + W (n))Y (n + k)]= E[Y (n)Y (n + k)] + E[W (n)Y (n + k)]} {{ } } {{ }=R Y Y (k)=0= R Y Y (k)R XX (k) = E[X(n)X(n + k)]In this case, (2) is given by⎡ ⎤0.14341⎢ ⎥⎣4. ⎦0= E[(Y (n) + W (n))(Y (n + k) + W (n + k))]= E[Y (n)Y (n + k)] + E[W (n)W (n + k)]= R Y Y (k) + R W W (k)= R Y Y (k) + 1 4 δ(k)=⎡⎤ ⎡ ⎤1 1 0 0 . . . 0 0 h(−M)4 1 11 0 . . . 0 04 4 .0 1 11 . . . 0 04 4 h(−1). ..·h(0). (3)1 1⎢0 0 . . . 1 0h(1)4 4 ⎥ ⎢ ⎥⎣10 0 . . . 0 1 1 ⎦ ⎣4 4 . ⎦10 0 . . . 0 0 h(M)412

Solving (3) yields⎡ ⎤h(−M).h(−1)h(0)h(1)⎢ ⎥⎣ . ⎦h(M)=⎡⎤1 1 0 0 . . . 0 04 1 11 0 . . . 0 04 4 0 1 11 . . . 0 04 4 . ..1 1⎢0 0 . . . 1 04 4 ⎥⎣10 0 . . . 0 1 1 ⎦4 410 0 . . . 0 041−1 ⎡⎤0.1434.1⎢ ⎥⎣4. ⎦0- Causal finite Wiener filter with length of M + 1:The causal Wiener filter is of the formŶ c (n) =M∑h(m)X(n − m)m=0Applying the orthogonality principle, [h(0), h(1), . . . , h(M)] T satisfies the linearequation⎡ ⎤ ⎡⎤ ⎡ ⎤R XY (0) R XX (0) R XX (1) . . . R XX (M) h(0)R XY (1)⎢ ⎥⎣ . ⎦ = R XX (1) R XX (0) . . . R XX (M − 1)h(1)⎢⎥ ⎢ ⎥⎣ .⎦ ⎣ . ⎦ .R XY (M) R XX (M) R XX (M − 1) . . . R XX (0) h(M)Inserting the values yields⎡ ⎤ ⎡⎤ ⎡ ⎤31 1 0 . . . 0 0 h(0)44 1141 1 . . . 0 04 4 h(1)=⎢ ⎥ ⎢..⎥ ⎢.⎥⎣0. ⎦ ⎣10 0 0 1 1 ⎦ ⎣h(M − 1) ⎦4 410 0 0 0 0 h(M)41(4)Solving (4) yields⎡ ⎤ ⎡⎤h(0) 1 1 0 . . . 0 04 h(1)11 1 . . . 0 04 4 ⎢.=⎥ ⎢..⎥⎣h(M − 1) ⎦ ⎣10 0 0 1 1 ⎦4 41h(M) 0 0 0 041−1 ⎡341⎢ 4⎤.⎢ ⎥⎣0. ⎦0b) Calculate the filter coefficients for M = 1. Compute the mean-square estimationerrors resulting when using both filters.3

- Non-causal finite Wiener filter with finite length 2M + 1, M = 1:When M = 1, (3) reduces to⎡ ⎤ ⎡ ⎤ ⎡ ⎤11 1 0 h(−1)44⎣ 3⎦ = ⎣ 11 1 ⎦ · ⎣ h(0) ⎦ .4 40 1 1 h(1)4Solving yields414⎡ ⎤h(−1)⎣ h(0) ⎦ =h(1)=⎡⎣⎡⎣15− 2 114 7 14− 2 8− 2 7 7 71− 2 1514 7 14⎤114 5 ⎦7 114⎤ ⎡⎦ ⎣The mean-square estimation error is calculated to beE [( Y (n) − Ŷc(n) ) 2]M∑= R Y Y (0) − h(m)R XY (m).m=−MWhen M = 1, the mean-square estimation error readsE[(Y (n) − Ŷ (n))2 ] = R Y Y (0) −∑+1m=−1143414⎤⎦h(m)R XY (m)= 3 4 − [ 114 · 14 + 5 7 · 34 + 114 · 14 ]= 528 .Comment:Using the same method, we calculated different non-causal Wiener filters withM = 0, 1, 2, .., 10. Figure (1) shows that as M increases the mean-squareestimation error curve (marked by ∗) converges to a stable level that coincideswith the MSE of the non-causal Wiener filter. It can be concluded that thefinite Wiener filter with M = 3 provides a good approximation of the noncausalWiener filter.- Causal finite Wiener filter with finite length 2M + 1, M = 1:When M = 1, (4) reduces to[ 3[ ] [ ]4 111 =4 h(0)1.2]1 h(1)4Thus the coefficient is given by[ ] h(0)h(1)==4[ 16− 415 15− 4 1615 15[ 11]15 1 .15] [ 34 12]

0.19Non−causalCausalPSfrag replacementsMean-square estimation error0.1850.180 1 2 3 4 5 6 7 8 9 10MFigure 1: Mean-square estimation error for the non-causal and the causal finite Wienerfilters versus M.The mean-square estimation error is calculated to beE [( Y (n) − Ŷc(n) ) 2]M∑= R Y Y (0) − h(m)R XY (m).m=0When M = 1, the mean-square estimation error isE [( Y (n) − Ŷc(n) ) 2]= R Y Y (0) −1∑h(m)R XY (m)m=0= 3 4 − [ 1115 · 34 + 115 · 1 ]4= 1160 .Comments:The causal Wiener filters, with M = 0, 1, 2, . . . , 10 are calculated as well. Thecurve marked with circles in Fig (1) represents the mean-square estimationerror M. We may observe similar situation that the mean-square estimationerror stabilizes after M ≥ 3. Hence the optimal estimator that can be achievedis the Wiener filter with length of 4, i.e. M = 3.By the way it can be also observed that the non-causal estimators performedbetter than the causal ones, in the sense that the mean-square estimationerrors caused by the non-causal filters are always smaller than (when M ≥ 1)or equal (when M = 0) to that of causal filters, i.e.,E [( Y (n) − Ŷ (n)) 2]≤ E[(Y (n) − Ŷ c (n) ) 2].5