Notes on Poisson Regression and Some Extensions

Moments.var(Y ) = µE(Y ) = µThe mean rateThe variance mean identityHow well does the mean/variance equivalence hold in the example data above? Obtain theobserved proportions of counts and apply the usual formula for the mean and variance of arandom variable. We find, as before, E(Y ) = 0.899 and var(Y ) = 0.860, which are close enoughfor most government work.Models and Estimation.We usually let µ depend on data through a loglinear model , so thatlog(µ i ) = x ′ iβThe log likelihood is written aslog L =n∑ {x′i β − exp(x ′ iβ) − log Γ(y i + 1) }i=1We can ignore the last term in this expression as it does not involve parameters. In other words,the log L evaluated without this term differs from the log L above by an additive constant that isindependent of the model parameters.Exercise 1: Suppose we are interested in finding the MLE of µ (i.e., without covariates). Showthat the log likelihood is simplyn∑log L = log µ y i − nµand that the MLE is ̂µ = ∑ y i /n by solving the first-order conditions for an MLE.Exercise 2: Use the second-order conditions for a maximum to show that the variance of̂µ = ̂µ 2 / ∑ y i .Exercise 3: Develop a least-squares estimator for µ by (1) devising a appropriate loss function(i.e., sum of squared deviations around the expected value), and (2) solving the normal equationfor µ.i=1Numerical Methods. (Please skip this if you are not interested in gory details underlying theblack box of most statistical programs.) The score vector and Hessian matrix areandu = X ′ (y − m)H = −X ′ MX,where m = exp(x ′ îβ) and M = diag(m)We can apply Newton-Raphson or IRLS to this problem. The NR iteration steps arêβ (t) = ̂β (t−1) [− H (t−1)] −1U (t−1) .2