an introduction to generalized linear models - GDM@FUDAN ...

We also need expressions for the expected value and variance of the derivatives
of the log-likelihood function. From (3.3), the log-likelihood function for
a distribution in the exponential family is
l(θ; y) =a(y)b(θ)+c(θ)+d(y).
The derivative of l(θ; y) with respect to θ is
dl(θ; y)
U(θ; y) =
dθ = a(y)b′ (θ)+c ′ (θ).
The function U is called the score statistic and, as it depends on y, it can
be regarded as a random variable, that is
Its expected value is
U = a(Y )b ′ (θ)+c ′ (θ). (3.13)
E(U) =b ′ (θ)E[a(Y )] + c ′ (θ).
From (3.9)
E(U) =b ′ �
(θ) − c′ (θ)
b ′ �
+ c
(θ)
′ (θ) =0. (3.14)
The variance of U is called the information and will be denoted by I. Using
the formula for the variance of a linear transformation of random variables
(see (1.3) and (3.13))
Substituting (3.12) gives
I = var(U) = � b ′ (θ) 2� var[a(Y )].
var(U) = b′′ (θ)c ′ (θ)
b ′ (θ) − c′′ (θ). (3.15)
The score statistic U is used for inference about parameter values in generalized
linear models (see Chapter 5).
Another property of U which will be used later is
var(U) =E(U 2 )=−E(U ′ ). (3.16)
The first equality follows from the general result
var(X) =E(X 2 ) − [E(X)] 2
for any random variable, and the fact that E(U) = 0 from (3.14). To obtain
the second equality, we differentiate U with respect to θ; from (3.13)
U ′ = dU
dθ = a(Y )b′′ (θ)+c ′′ (θ).
Therefore the expected value of U ′ is
© 2002 by Chapman & Hall/CRC
E(U ′ ) = b ′′ (θ)E[a(Y )] + c ′′ (θ)
= b ′′ �
(θ) − c′ (θ)
b ′ �
+ c
(θ)
′′ (θ) (3.17)
= −var(U) =−I