Solutions to Assignment 3

University of Western Australia School of Mathematics & Statistics ajb/05SS225 Statistical Science 530.225Solutions to Assignment 31. (a) Since X has mean np = 10, 000 × 0.1 = 1000 and variance np(1 − p) = 10, 000 × 0.1 × 0.9 = 900,it is approximately Normal with the same mean µ = 1000 and variance σ 2 = 900. If Y denotesa N(1000, 900) random variable and Z is a N(0, 1) r.v. thenw = P{950 ≤ X ≤ 1050} ≈ P{950 ≤ Y ≤ 1050}= P{ 950 − µ ≤ Y − µ ≤ 1050 − µ }σ σ σ950 − 1000 1050 − 1000= P{ ≤ Z ≤ }3030= P{− 5 3 ≤ Z ≤ 5 3 }.From tables, P{Z ≤ 5 } ≈ 0.9522 so that P{Z > 5 } ≈ 1 − 0.9522 = 0.0478 so that3 3P{− 5 ≤ Z ≤ 5 } ≈ 1 − 2 × 0.0478 = 0.9044. That is, w ≈ 0.9044.3 3(b) We notice that 950 ≤ X ≤ 1050 if and only if |X − 1000| ≤ 50. Since 1000 is the expectedvalue of X we can apply Chebyshev’s inequality to getso thatP{|X − 1000| > 50} ≤ σ250 2 = 9 25 = 0.36w = P{950 ≤ X ≤ 1050} = 1 − P{|X − 1000| > 50}≥ 1 − 0.36 = 0.64that is, w ≥ 0.64. This is not a very accurate bound on the value of w.2. (a)S n = s 0 R 1 R 2 . . . + R n=n∏s 0 R i .i=1(b) We haveL = log S n = log s 0 + log R 1 + . . . + log R nn∑= log s 0 + log R i .Since R 1 , R 2 , . . . are i.i.d., clearly log R 1 , log R 2 , . . . are i.i.d. random variables. So by the CentralLimit Theorem, when n is large the distribution of ∑ ni=1 log R i is approximately Normal, withmean nE[log R 1 ] and variance nvar(log R 1 ). Since s 0 is a constant, it follows that L is alsoapproximately Normal with mean µ = log s 0 + nE[log R 1 ] and variance nvar(log R 1 ).i=1

3. (a) If X has this distribution, then its mean isE δ [X] =∫ δ0xf(x)dx= 1 ∫ δ2x 2 dxδ 2= 2 3 δ.Since there is only one parameter, the method of moments is to equate the population andsample means, i.e. to solveE δ [X] = ¯X,that is, to solvewhich yields(b) The likelihood is023 δ = ¯X̂δ = 3 2 ¯X.L(δ) = f(x 1 , . . . , x n ; δ)= f(x 1 ; δ) . . . f(x n ; δ){ 2 n ∏ n= δ 2n i=1 x i if 0 ≤ x i ≤ δ for all i0 otherwise{ 2 n ∏ n=δ 2n i=1 x i if δ ≥ max i x i0 otherwiseassuming x i ≥ 0 for all i. The likelihood is decreasing on the interval [m, ∞) where m = max i x i .Hence it achieves its maximum at δ = m. That is, the maximum likelihood estimator iŝδ = max i X i .4. (a) We have to minimise the sum of squared deviations between the observed breaking strains y kand the predicted breaking strains kβ. That is, we minimiseS(β) =n∑(y k − kβ) 2k=1as a function of β. Differentiating we getn∑S ′ (β) = (−2k)(y k − kβ)k=1= −2 ∑ ky k + 2β ∑kkk 2 .Equating this to zero yieldŝβ =∑ nk=1 ky k∑ nk=1 k2 .Since the second derivative S ′′ (β) is positive, we have found a minimum of the function S,confirming that this is the least squares estimator.2

(b) Consider ̂β as a function of the random variables Y k . We have∑̂β = ∑ k kY kk k2= ∑ ka k Y kwhere a k = k/ ∑ nj=1 j2 for each k = 1, . . . , n. Since E[Y k ] = kβ, we haveE[̂β] = E[ ∑ ka k Y k ]= ∑ k= ∑ k= ∑ ka k E[Y k ]a k kβk∑ n kβj=1j2=∑ nk=1 βk2 ∑ nj=1 j2= β.That is, the estimator is unbiased.5. The likelihood isThe log likelihood isL(λ, µ) = f(x 1 , . . . , x n )= f(x 1 ) . . . f(x n )= λn ∏ ne −λ|x i−µ|2 n= λni=1n∑2 exp{−λ |x n i − µ|}.i=1log L(λ, µ) = n log λ − n log 2 − λn∑|x i − µ|.This is a continuous function of λ and µ. It is differentiable with respect to λ, and also differentiablewith respect to µ except when µ = x i for some i. Sinceddx |x| = ⎧⎨⎩we find that, for any µ which satisfies µ ≠ x i for all i,i=1−1 if x < 0+1 if x > 0undefined if x = 0∂n∑∂µ L(λ, µ) = λ sign(x i − µ)where sign(x) = −1 for x < 0 and sign(x) = +1 for x > 0. Nown∑sign(x i − µ) = k − (n − k) = 2k − ni=13i=1

where k is the number of values x i satisfying x i > µ. This number is positive if k < n/2 and negativeif k > n/2. Since n is odd, say n = 2m + 1, this means that ∂ L(λ, µ) is positive when k ≤ m and∂µnegative when k ≥ m.Suppose we rank the values x i in increasing order,x [1] < x [2] < . . . < x [n] .Then L(λ, µ) is an increasing function of µ for µ < x [m] and a decreasing function of µ for µ > x [m] .Hence the likelihood is maximised at̂µ = x [m] .That is, the maximum likelihood estimator of µ is the median of the values x i .6. First we haveNowwhileE[(X 1 − X) 2 ] = E[X 2 1 − 2X 1X + (X) 2 ]= E[X 2 1 ] − 2E[X 1 X] + E[(X) 2 ].E[X 2 1 ] = var(X 1) + (E[X 1 ]) 2 = σ 2 + µ 2E[(X) 2 ] = var(X) + (E[X]) 2 = σ2n + µ2 .It remains to calculate E[X 1 X]. The variables X 1 and X are not independent. Writingwe getX = 1 nn∑X j = 1 n (X 1 +j=1n∑X j )j=2E[X 1 X] = 1 n∑n E[X2 1 + X 1 X j ]j=2= 1 n E[X2 1 ] + 1 n E[X 1]E[because X 1 is independent of ∑ nj=2 X j. This reduces ton∑X j ]j=2E[X 1 X] = 1 n (µ2 + σ 2 ) + 1 µ(n − 1)µnSo finally we get= µ2n + σ2 (n − 1)µ2+n n= µ 2 + σ2n .E[(X 1 − X) 2 ] = (σ 2 + µ 2 ) − 2(µ 2 + σ2n ) + σ2n + µ2= σ 2 − σ2n= n − 1n σ2 .4

Finally considerV = cn∑(X i − X) 2 .Clearly E[(X i − X) 2 ] = E[(X 1 − X) 2 ] for each i, by symmetry. Hence we haveE[V ] = cIt follows that taking c = 1/(n − 1), that is,i=1n∑E[(X i − X) 2 ]i=1= ncE[(X 1 − X) 2 ]= (n − 1)cσ 2 .V = 1n − 1n∑(X i − X) 2 ,gives E[V ] = σ 2 , that is, V is an unbiased estimator of σ 2 .7. (a)(b)i=1L(p) = P{X 1 = x 1 , . . . , X n = x n }n∏= P{X i = x i }=i=1n∏p(1 − p) x i−1i=1= p n (1 − p) P i x i−n .U(p) = ∂ log L(p)∂p= ∂ n∑∂p [n log p + log(1 − p)( x i − n)]= n ∑ np − i=1 x i − n.1 − p(c) The likelihood is differentiable for all values of p. Equating the score to zero yieldsor equivalentlyi.e.∑n np = i=1 x i − n1 − p1 − pp∑ ni=1=x i − nn1p − 1 = ¯x − 1so that ̂p = 1/x. The second derivative of the log likelihood is U ′ (p) = − n − whichp 2is negative, confirming that this is a maximum of the likelihood. Thus the MLE is ̂p = 1/x.5i=1P ni=1 x i−n(1−p) 2

(d) Since E[X i ] = 1/p we getE p [U(p, X)] = E[ n ∑ np − i=1 X i − n]1 − p= n p − n 1 − n p1 − p= n p − n − npp(1 − p)= 0.[At this point there was a typing error:E p [U(q, X)].”]If q ≠ p we haveit should have read “If q ≠ p, find the value ofE p [U(q, X)] = E[ n ∑ nq − i=1 X i − n]1 − q= n q − n 1 − n p1 − q= n n(1 − p)−q p(1 − q) .(e) Since we already calculated U ′ (p) it is easiest to use this form, givingI n (p) = E[−U ′ (p)]= E[ n p 2 + ∑ ni=1 X i − n(1 − p) 2 ]= n p + n 1 − n p 2 (1 − p) 2= n p + n2 p(1 − p)n=p 2 (1 − p) .(f) The maximum likelihood estimator ̂p is asymptotically Normally distributed with mean p andvariance 1/I n (p) = p 2 (1 − p)/n.6