SOLUTION FOR HOMEWORK 3, STAT 4352 Welcome to your third ...

Given this information, the posterior distribution of Θ given the observation X is
=
f Θ|X (θ|x) = fΘ (θ)f X|Θ (x|θ)
f X (x)
Γ(n + α + β)
Γ(x + α)Γ(n − x + β) θx+α−1 (1 − θ) (n−x+β)−1 .
The algebra leading to the last equality is explained on page 345.
Now you can realize that the posterior distribution is again Beta(x+α, n−x+β). There
are two consequences from this fact. First, by a definition, if a prior density and a posterior
density are from the same family of distributions, then the prior is called conjugate. This
is the case that Bayesian statisticians like a lot because this methodologically support the
Bayesian approach and also simplifies formulae. Second, we know a formula for the mean of
a beta RV, and using it we get the Bayesian estimator
ˆΘB = E(Θ|X) =
X + α X + α
=
(α + X) + (n − X + β) α + n + β
Now we actually can consider the exercise at hand. A general remark: Bayesian estimator
is typically a linear combination of the prior mean and the MLE estimator with weights
depending on variances of these two estimates. In general, as n → ∞, Bayesian estimator
approaches the MLE.
Let us check that this is the case for the problem at hand. Write,
Now, if we denote
ˆΘB = X
n
we get the wished presentation
n α
+
α + β + n α + β
w :=
n
α + β + n ,
ˆΘ = w ¯ X + (1 − w)θ0.
α + β
α + β + n .
where θ0 = E(Θ) = α/(α + β) is the prior mean of Θ.
Now, the problem at hand asks us to work a bit further on the weight. The variance of
the beta RV Θ is
Well, it is plain to see that
Then a simple algebra yields
V ar(Θ) := σ 2 0 =
θ0(1 − θ0) =
αβ
(α + β) 2 (α + β + 1) .
αβ
(α + β) 2.
σ 2 0 = θ0(1 − θ0)
α + β + 1
6