Chapter 4 Linear Models

Chapter 4Linear ModelsA.T.S.P.Andrei Trusculescu gr. 1.2

Linear modelsIntroductionThe determination of the MVU estimator is in generala difficult task. It is fortunate, however, that a largenumber of signal processing estimation can berepresented by a model that allows us to easilydetermine this estimator.To find the optimal estimator is in structuring theproblem in the linear model form to take advantage ofits unique properties.

Definition and PropertiesThe linear models has been discussed already in line fittingtingproblem in the example 3.7 (to fit a straight line throughnoise corrupted data).Model of the data :

Where w[n] ] is WGN (White Gaussian Noise) and the slope Band intercept A where to be estimate.In matrix notation the models is written more compactly as :

H is a known matrix of dimension Nx2 and is referred as theobservation matrix .In defining the linear model we assume that the noise vector r isGaussian although other authors use the term more generallyfor any noise PDF.From chapter 3 the MVU estimator can be determined if theequality constraints of the CRLB theorem are satisfied.So from chapter 3, Θ=g(x) ) will be the MVU estimator if :The covariance matrix of Θ will be I -1 (Θ).

For A a symmetric matrix we have :And assuming that H T H is invertible :Which is exactly :

The MVU estimator of Θ and its covariance matrixTheorem : Minimum Variance Unbiased Estimator for theLinear ModelAs illustrated in the figure bellow it is clear that an infinitenumber of choices can be made for A and B that will result inthe same observations or given a noiseless x, Θ is not unique.Although rarely occurring in practice, this degeneracysometimes nearly occurs when H T H is ill-conditioned.

x[n]=]=A+B+w[n]] = A+B2=A+B, where x is an Nx1 vector of observation, H is a knownNxp observation matrix (with N>p) and rank p, Θ is a px1vector of parameters to be estimated, and w is Nx1 vector withPDF N(0,Lσ 2 I) , then the MVU estimator isand the covariance matrix of Θ isThe Gaussian nature of the MVU estimator for the linear modelsallows to determine the exact statistical performance if desired.