GLS (Generalized Least Squares) - Paul Johnson Homepage
GLS (Generalized Least Squares) - Paul Johnson Homepage
GLS (Generalized Least Squares) - Paul Johnson Homepage
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
In matrix form, the representation of WLS/<strong>GLS</strong> is<br />
S(ˆb) = (y − ŷ) ′ W (y − ŷ) (12)<br />
and, if you write that out, the sum of squares in a <strong>GLS</strong> framework is a rather more complicated<br />
thing. All of those off-diagonal w ij ’s make the number of terms multiply.<br />
⎡<br />
⎤ ⎡<br />
⎤<br />
w 11 w 12 · · · w 1N y 1 − ŷ 1<br />
[ ]<br />
w 12 w 22 w 2N<br />
y 2 − ŷ 2<br />
y1 − ŷ 1 y 2 − ŷ 2 · · · y N − ŷ N ⎢<br />
⎣ .<br />
.. .<br />
⎥ ⎢<br />
⎥<br />
. ⎦ ⎣ . ⎦<br />
w N1 w N2 · · · w NN y N − ŷ N<br />
= [ ]<br />
y 1 − ŷ 1 y 2 − ŷ 2 · · · y N − ŷ N ⎢<br />
⎣<br />
⎡<br />
w 11 (y 1 − ŷ 1 ) +w 12 (y 2 − ŷ 2 ) · · · +w 1N (y N − ŷ N )<br />
w 12 (y 1 − ŷ 1 ) +w 22 (y 2 − ŷ 2 ) +w 2N (y N − ŷ N )<br />
.<br />
.<br />
.. .<br />
w N1 (y 1 − ŷ 1 ) +w N2 (y 2 − ŷ 2 ) · · · +w NN (y N − ŷ N )<br />
⎤<br />
⎥<br />
⎦<br />
⎡<br />
=<br />
⎢<br />
⎣<br />
w 11 (y 1 − ŷ 1 ) 2 ⎤<br />
+w 12 (y 2 − ŷ 2 )(y 1 − ŷ 1 ) · · · +w 1N (y N − ŷ N )(y 1 − ŷ 1 )<br />
w 12 (y 1 − ŷ 1 )(y 2 − ŷ 2 ) +w 22 (y 2 − ŷ 2 ) 2 +w 2N (y N − ŷ N )(y 2 − ŷ 2 )<br />
.<br />
.. .<br />
⎥<br />
.<br />
⎦<br />
w N1 (y 1 − ŷ 1 )(y N − ŷ N ) +w N2 (y 2 − ŷ 2 )(y N − ŷ N ) · · · +w NN (y N − ŷ N ) 2<br />
w 11 (y 1 − ŷ 1 ) 2 + w 12 (y 2 − ŷ 2 )(y 1 − ŷ 1 ) + · · · + w 1N (y N − ŷ N )(y 1 − ŷ 1 )<br />
= +w 12 (y 1 − ŷ 1 )(y 2 − ŷ 2 ) + w 22 (y 2 − ŷ 2 ) 2 + · · · + w 2N (y N − ŷ N )(y 2 − ŷ 2 )<br />
+w N1 (y 1 − ŷ 1 )(y N − ŷ N ) + w N2 (y 2 − ŷ 2 )(y N − ŷ N ) + · · · + w NN (y N − ŷ N ) 2<br />
Supposing a linear model:<br />
N∑ N∑<br />
= w ij (y i − ŷ i )(y j − ŷ j ) (13)<br />
i=1 j=1<br />
ŷ = Xˆb (14)<br />
The normal equations (the name for the first order conditions) are found by setting the derivatives<br />
of the Sum of <strong>Squares</strong> with respect to the parameters equal to 0.<br />
for each parameter ˆb j :<br />
∂<br />
∂ˆb j<br />
[(y − ŷ) ′ W (y − ŷ)] = 0<br />
If you insert ŷ = Xb in there and do the math (or look it up in a book :) )<br />
(X ′ W X)ˆb − X ′ W y = 0<br />
The WLS/<strong>GLS</strong> estimator is the solution:<br />
ˆb = (X ′ W X) −1 X ′ W y (15)<br />
3