GLS (Generalized Least Squares) - Paul Johnson Homepage

More documents

Recommendations

Info

$Simpson's in LATEX Beamer Presentations$

⎡ V ar(e) = ⎢ ⎣ In weighted least squares, we use estimates of σ 2 e i σe 2 ⎤ 1 0 0 0 0 σe 2 2 0 0 . .. 0 . 0 .. 0 0 σe 2 ⎥ N−1 0 ⎦ 0 0 0 0 σe 2 N as weights. (6) S(ˆb) = N∑ i=0 1 ̂σ 2 e i (y i − ŷ i ) 2 = Consider the WLS problem. If the heteroskedastic case occurs N∑ w i (y i − ŷ i ) 2 (7) i=0 N∑ N∑ S(ˆb) = w i (y i − ŷ i ) 2 = ( √ w i y i − √ w i ŷ i ) 2 (8) i=1 i=1 = w 1 (y 1 − ŷ 1 ) 2 + w 2 (y 2 − ŷ 2 ) 2 + · · · + w N (y N − ŷ N ) 2 (9) 3 Generalized Least Squares (GLS) Suppose, generally, the Variance/Covariance matrix of residuals is ⎡ σ1 2 Cov(e 1 , e 2 ) Cov(e 1 , e 2 ) · · · Cov(e 1 , e N ) Cov(e 1 , e 2 ) σ2 2 σ3 2 V = . . .. ⎢ ⎣ σN−1 2 Cov(e 1 , e N−1 ) Cov(e 1 , e N ) Cov(e 1 , e N−1 σN 2 ⎤ ⎥ ⎦ (10) You can factor out a constant σ 2 if you care to (MM&V do, p. 51). If all the off diagonal elements of V are set to 0, then this degenerates to a problem of heteroskedasticity, know as Weighted Least Squares (WLS). If the off diagonal elements of V are not 0, then there can be correlation across cases. In a time series problem, that amounts to “autocorrelation.” In a cross-sectional problem, it means that various observations are interrelated. The “sum of squared errors” approach uses this variance matrix to adjust the data so that the residuals are homoskedastic or that the cross-unit correlations are taken into account. Let W = V −1 (11) The idea behind WLS/GLS is that there is a way to transform y and X so that the residuals are homogeneous, some kind of weight is applied. 2
In matrix form, the representation of WLS/GLS is S(ˆb) = (y − ŷ) ′ W (y − ŷ) (12) and, if you write that out, the sum of squares in a GLS framework is a rather more complicated thing. All of those off-diagonal w ij ’s make the number of terms multiply. ⎡ ⎤ ⎡ ⎤ w 11 w 12 · · · w 1N y 1 − ŷ 1 [ ] w 12 w 22 w 2N y 2 − ŷ 2 y1 − ŷ 1 y 2 − ŷ 2 · · · y N − ŷ N ⎢ ⎣ . .. . ⎥ ⎢ ⎥ . ⎦ ⎣ . ⎦ w N1 w N2 · · · w NN y N − ŷ N = [ ] y 1 − ŷ 1 y 2 − ŷ 2 · · · y N − ŷ N ⎢ ⎣ ⎡ w 11 (y 1 − ŷ 1 ) +w 12 (y 2 − ŷ 2 ) · · · +w 1N (y N − ŷ N ) w 12 (y 1 − ŷ 1 ) +w 22 (y 2 − ŷ 2 ) +w 2N (y N − ŷ N ) . . .. . w N1 (y 1 − ŷ 1 ) +w N2 (y 2 − ŷ 2 ) · · · +w NN (y N − ŷ N ) ⎤ ⎥ ⎦ ⎡ = ⎢ ⎣ w 11 (y 1 − ŷ 1 ) 2 ⎤ +w 12 (y 2 − ŷ 2 )(y 1 − ŷ 1 ) · · · +w 1N (y N − ŷ N )(y 1 − ŷ 1 ) w 12 (y 1 − ŷ 1 )(y 2 − ŷ 2 ) +w 22 (y 2 − ŷ 2 ) 2 +w 2N (y N − ŷ N )(y 2 − ŷ 2 ) . .. . ⎥ . ⎦ w N1 (y 1 − ŷ 1 )(y N − ŷ N ) +w N2 (y 2 − ŷ 2 )(y N − ŷ N ) · · · +w NN (y N − ŷ N ) 2 w 11 (y 1 − ŷ 1 ) 2 + w 12 (y 2 − ŷ 2 )(y 1 − ŷ 1 ) + · · · + w 1N (y N − ŷ N )(y 1 − ŷ 1 ) = +w 12 (y 1 − ŷ 1 )(y 2 − ŷ 2 ) + w 22 (y 2 − ŷ 2 ) 2 + · · · + w 2N (y N − ŷ N )(y 2 − ŷ 2 ) +w N1 (y 1 − ŷ 1 )(y N − ŷ N ) + w N2 (y 2 − ŷ 2 )(y N − ŷ N ) + · · · + w NN (y N − ŷ N ) 2 Supposing a linear model: N∑ N∑ = w ij (y i − ŷ i )(y j − ŷ j ) (13) i=1 j=1 ŷ = Xˆb (14) The normal equations (the name for the first order conditions) are found by setting the derivatives of the Sum of Squares with respect to the parameters equal to 0. for each parameter ˆb j : ∂ ∂ˆb j [(y − ŷ) ′ W (y − ŷ)] = 0 If you insert ŷ = Xb in there and do the math (or look it up in a book :) ) (X ′ W X)ˆb − X ′ W y = 0 The WLS/GLS estimator is the solution: ˆb = (X ′ W X) −1 X ′ W y (15) 3
Page 1: GLS (Generalized Least Squares) Pau
Page 5: = (X ′ X) −1 X ′ (σ 2 eI)X(X

GLS (Generalized Least Squares) - Paul Johnson Homepage

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?