Generalized linear models in R

www.nr.no
Generalized linear models in R
Magne Aldrin, Norwegian Computing Center and the
University of Oslo
Sharp workshop, Copenhagen, October 2012

Linear regression
Model
y = β 0 + β 1 x 1 + β 2 x 2 + · · · + β p x p + ε
• y: response variable, continious
• x 1 , · · · x p : p explanatory variables, predictor variables, covariates
• β 0 : intercept
• β 1 , · · · β p : regressions coeffisients
• ε: error term, E(ε) = 0, V ar(ε) = σ 2
1

Estimation
• The β’s are unknown, and must be estimated from observations
• n observations of y i , x 1i , . . . x pi
• Estimation by Ordinary Least Squares
⋆ Minimize ∑ i (y i − ŷ i ) 2 ,
⋆ where ŷ i = ˆβ 0 + ˆβ 1 x 1i + ˆβ 2 x 2i + · · · + ˆβ p x pi
• If ε is normal distributed (Gaussian), then OLS = Maximum
Likelihood (ML)
2

Generalized Linear Models - GLM
• y ∼ Distributed with mean µ and perhaps an additional parameter
• h(µ) = η
• η = β 0 + β 1 x 1 + β 2 x 2 + · · · + β p x p
• η: linear predictor
• h: link fuction
• Estimation by maximum likelihood
3

Gaussian regression/linear regression
• Continious, symmetric response
• y ∼ N(µ, σ 2 )
• µ = η : identity link
4

Gamma regression
• Continous, positive, skew response
• y ∼ Gamma(expectation µ, parameter2)
• log(µ) = η : log link
• µ = exp(η)
5

Binary logistic regression
• Binary response, take values 0 or 1, binomial with n = 1 trial
• P (y = 1) = p = E(y) = µ
• y ∼ Bin(n = 1,p)
• logit(p) = log(p/(1 − p)) = η : logit link
• p = exp(η)/(1 + exp(η)) : logistic inverse link
6

Binomial logistic regression
• Binomial response, n trials, take values 0, 1, . . . , n
• y ∼ Bin(n,p)
• logit(p) = log(p/(1 − p)) = η : logit link
• p = exp(η)/(1 + exp(η)) : logistic inverse link
7

Poisson regression
• Count response, take values 0, 1, 2, . . .
• y ∼ Poisson(λ = µ)
• log(µ) = η : log link
• µ = exp(η)
8

Ex: Ozone data - Gaussian regression
• Ozone - O3 is the response variable
• 9 expanatory variables (wind speed, humidity, temperature ...)
• 330 observation
> require(faraway)
> data(ozone)
>
> summary(ozone)
O3 vh wind humidity
Min. : 1.00 Min. :5320 Min. : 0.000 Min. :19.00
1st Qu.: 5.00 1st Qu.:5690 1st Qu.: 3.000 1st Qu.:47.00
Median :10.00 Median :5760 Median : 5.000 Median :64.00
Mean :11.78 Mean :5750 Mean : 4.848 Mean :58.13
3rd Qu.:17.00 3rd Qu.:5830 3rd Qu.: 6.000 3rd Qu.:73.00
Max. :38.00 Max. :5950 Max. :11.000 Max. :93.00
9

temp ibh dpg ibt
Min. :25.00 Min. : 111.0 Min. :-69.00 Min. :-25.0
1st Qu.:51.00 1st Qu.: 877.5 1st Qu.: -9.00 1st Qu.:107.0
Median :62.00 Median :2112.5 Median : 24.00 Median :167.5
Mean :61.75 Mean :2572.9 Mean : 17.37 Mean :161.2
3rd Qu.:72.00 3rd Qu.:5000.0 3rd Qu.: 44.75 3rd Qu.:214.0
Max. :93.00 Max. :5000.0 Max. :107.00 Max. :332.0
vis
doy
Min. : 0.0 Min. : 33.0
1st Qu.: 70.0 1st Qu.:120.2
Median :120.0 Median :205.5
Mean :124.5 Mean :209.4
3rd Qu.:150.0 3rd Qu.:301.8
Max. :350.0 Max. :390.0
10

pdf("PAIRSozone.pdf")
> pairs(ozone)
> dev.off()
11

12
O3
5300 5800
●●●● ●
● ●
● ●● ● ●
●
●●●●
● ● ● ● ●
●
●
●
●
● ● ●
●● ●● ●
●
● ●●
●●●
● ●● ●
●
●
●●
●
● ●●
●
● ●●
● ●● ●
●
●
● ●● ●
● ● ● ●
●●
●
● ● ● ●
● ●
● ●
● ● ●
●
●●● ●
●
● ● ●●
●
● ● ● ● ●
●
●
●
● ● ● ● ●
●
●
●
●
●
●
● ● ●
●●
●
●
●●
●●
●● ●
● ●●
● ● ●
●
●
● ● ●●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
● ●●●●
●
●
●
●
●
● ● ●●
● ● ●
●
●
●●
● ●● ● ●●
●
● ●● ●
●
● ●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ● ●
●
●
● ● ●
●
●
●
●
●
●
●
● ●
●
● ●
●
●
●●
●
●● ●
● ● ●
●
●
● ●
●
●
● ● ●● ●
● ●
●
●
●
● ● ●
●
● ●●●
●
●
● ● ●
●
● ●●● ● ●
●
● ●● ●●●
● ●●
●
●
●
●● ●
●
● ● ●
●
●
●
● ●
●
●● ●
● ●
●
● ● ●
●
●
●● ●
●
●
●
● ●
●● ● ●●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
● ●●
● ● ● ● ●
● ●
●● ●
●
●
●
●
●
● ● ●
● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
●
●
● ●
● ●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
●● ●
● ●
●
●
●
● ●
●
● ●
●
●
● ● ●
● ●
● ●
● ● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
● ●
●
● ●
● ●
●
●
● ●
●
●
● ● ● ●
● ●
●
● ● ●●
●
●
●
●
●
●
●● ●
● ●
● ●
● ●
● ● ● ●
● ●
● ● ● ●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
● ● 20 60
● ● ● ●
●●
●
●
● ●
● ●
●
● ●
●●
● ●
●
● ●●
●
●
●
●
●●●●
● ●
●
● ●● ●
●
●
●
●
●
●
● ●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●● ●
●
●
● ● ● ●
● ●
●●
●
● ● ●
●● ●
●
●● ● ●
●
●
●● ●
● ● ●
●
●
●
●
● ●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
● ●
●
●
●
● ●
●● ●
●
●
●
● ● ●●
●
●
● ●●
●
●
● ● ●
●
●
●
●
● ●●
● ● ●
●
●
●
●● ●
● ●
● ● ● ●
●
●
●●
●
●
●●
● ●
● ●
●
● ●
● ●
● ● ●
● ● ● ●●
●●
●
●
● ●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
● ●
●
●
●
● ●
●
●
●
● ●● ●
● ● ● ●
●
●
●
● ●
●●
●
● ●
●
● ●
●
●●
●
●
●
●
●● ●
●
●
● ● ●
● ● ● ●
●
●
●
●●
● ●
● ●
●
●
●
●
●
●
●
●
● ●
● ● ● ● ●●
● ● ●
●● ●●●● ● ● ●
●● ●
● ● ●● ● ● ●
●
●
●●●
●● ●● ●
● ●● ●
●●● ● ●
● ●
●
●
● ●
● ● ●●● ●● ● ●● ●
●
●
●
● ●●
●
● ●
●
●
●●●
● ●
● ● ●
●
●
●
● ●
● ●
●
● ●● ● ● ●
● ● ● ●
●
●
●
● ● ●
●
●
●
● ● ● ● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●● ● ●●
● ● ●
●
●
●
●
●●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
● ●●● ●●
●
●
●● ●
● ●● ● ●
●
●
●
● ●
●
●
● ● ● ●●
●
● ●● ● ● ● ● ● ●●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
●●●
●
●
● ● ●
● ● ●
● ●
●
●
● ● ●
● ●
● ●
●●
●
● ● ● ●● ●
●
● ●
●
● ● ● ●● ● ● ●
●
●
●
● ● ●
●
●●●● ●
● ● ●
● ●●● ● ●●
● ●● ●●● ● ● ●
●
0 3000
●
● ● ●
●
●●
●
●●
●
●
●
●
●
●
●●
● ● ●
●●
● ●
●
●
●
●
● ●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●●● ●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
● ●
● ●
● ●
● ● ●●
● ● ●
● ● ●
●
●
●
● ● ●
●
●
●
●
●
●
● ● ●
● ● ●
● ●
●●
●
●
●
● ●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
● ● ●
● ●● ●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
● ●
●
●
●
●
● ●
● ● ●
●
●
●
● ● ●
● ● ●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ● ● ●●
●
●
●
● ●
●●
● ● ●
●● ●
●
●
● ● ●
● ● ●
●
●
● ●●
●
●
●
● ● ● ●●
●
● ●
●
●●
● ●●
● ●
●
●
●
●
● ● ●● ●●
● ●●
●
●
●
●
●
● ●●
●
●
●
● ●
●
●
● ●● ●
●●
●
●
●
● ● ●●
● ●
●
●
● ●
●
● ● ●
●
●●
●
● ●
● ●
●
● ● ●
● ● ● ●
●
●
●
●
●
●
● ●● ●
●●
●
●
●●
●●
● ● ● ●●●
● ●
● ●
● ●
●●
●
●
●
● ● ●
●
●
●
●
●
●
● ● ● ●●● ●●
●
●
●
●●
●
● ●
●
●
● ● ●
● ● ●● ● ●
●
● ●● ● ●
●
●
●
● ●
● ●●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●●
●●
●
●
●
●
●
● ● ● ●
●
●
● ●
●
●
●
●
● ● ●●
● ● ● ● ●● ●
●●
●
● ● ● ● ● ●
●●
● ● ●
●
●
●
● ● ●
●
●●●
●● ●
●
●
● ●● ●
●● ●
● ●● ●
● ●● ● ●
●
● ●
0 200
● ●● ●
●
● ● ●
●
●
●● ● ●
●●
● ● ●
● ● ● ●
●
●
●
● ●
●●
● ● ●
●● ●●●
●●
●
●● ●
●
●
●
●● ●●
● ●
● ●● ●
●
●
● ● ●●
●
●
●● ● ● ● ●
●
●
● ● ●
● ●●
●●● ●
● ● ● ● ●
●
● ● ●●
●
●
●
● ● ●
●
●
●
● ● ● ● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●● ●
●
●●
●
●
●
● ● ●●
●
●
● ●
●
● ● ● ●
●
●
●
●
●
●●
● ●●●●
●
●
● ●●● ●
●● ●●
● ● ●
● ●
●
●
● ● ● ●
●
●
● ●
● ● ● ● ●
● ●● ●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
● ● ●
● ● ●
●● ●
● ● ● ●
● ● ●
●
●●
●
●● ●●●● ●
●
●
● ●
● ● ● ● ● ● ●
● ● ● ●●
●
●
●● ●●
●●●
●●●
●
●
●
●●● ● ●
●● ●
●
●
● ●
●
●
●
● ●
●
●
● ●
●
●●
●
●
● ● ●
● ●
● ● ●
●
● ●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●● ●
●
● ●
●
●
●
●
● ● ●
●●
● ● ● ● ●
●
●
●● ● ●●
●● ●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
● ●● ●
● ●
●
●
●●
●●
●
● ● ●● ●● ●
●
●
●●
● ● ● ● ●●
● ● ●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
● ●●● ● ●
●● ● ● ●
●
●
●● ●
● ●
●
●
●
●●● ●
●
●
●
●
● ●
●
●
●
● ●● ●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
● ●●
●
●
● ●
●
●
●
●
● ● ●● ●●
●
●
● ● ● ●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
● ●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●● ●● ●
●
●
●●
● ●
●
●
●
●
●
● ● 50 300
0 20
● ●● ● ● ● ● ● ● ● ● ● ● ●
● ●● ●
●
●● ●
● ●● ●
●
● ● ● ● ● ● ●● ●● ● ● ● ● ● ●●● ● ● ● ● ●
●
● ● ●
●
● ● ●
● ●●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
● ● ● ● ●
● ● ●
● ● ● ● ●
● ● ● ● ●
●
●
● ● ● ●●●●
●
●
● ●●● ● ●●
● ● ●
● ● ● ● ● ● ● ●● ●●
● ● ● ● ●
●
●
●
●
● ● ●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ● ●
● ● ● ●● ●
● ●● ●
● ● ● ● ●
●
●●
●
●
●
● ● ●
● ●
● ● ● ●
●
●● ● ● ● ● ● ●
5300 5800
●● ● ● ● ●
● ●●
● ● ● ●
●●● ●● ●
●●
● ●
●
●
●
●
●
●
● ● ●●● ● ●
● ●●
●● ● ●● ●
●
●
● ● ●
● ●● ● ●
● ● ●●●
●
●
●
● ●●●
●
●
● ●●
●● ●
●
● ●●
● ●
● ●● ● ●
●
●● ● ●
●
● ● ● ●
●
● ●
●● ●
●
●
●
● ● ● ● ● ●
● ●
● ●
● ●
● ●
● ●
●
●
●
● ●● ●● ●
● ● ●
● ● ●●
●
●
● ● ● ●
●
●●
●
● ●
●● ●
●
●
●
●
● ●●●●
● ●
●●●●
●● ●
● ●
●
●
●
●●
●● ●
●
●●
●
● ● ● ●
● ● ●
● ●●●● ●
●● ●
●
●
●
● ●
●
●
● ● ●
●
● ●
●
●
●
● ●
●● ●
●
● ● ● ●●
● ●
● ●
●
●
●
● ●
●
●
● ●
● ●
●
● ●
●● ● ● ●
●
●
●
● ●
●
●
● ●
●
● ●
● ● ●●
●
●● ● ●
●
●
● ●● ●● ●● ●● ●● ● ● ●
● ● ● ●
● ●●
●
vh
●●
● ● ●
●
●
● ● ●
●
●
●
●●
●
● ●●●
●
● ●
● ● ●
●
●
● ● ●
●
●
●
● ●
● ●
● ●
●
●
●
● ● ●
● ● ●
●
●
●
●
●
●
●
●
●
●
● ●
● ● ● ●
●●
●●
●
●
●
●
●
●
● ●
●
● ●● ● ●
● ● ●
● ●
●
●
●
● ●●
● ●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
● ●●
●
●●
●
●
●
●
●
●
●
● ●
●
●● ●
●
●
●
●
● ●
● ●
●
●
●●
●● ●
●
●
● ●
● ●
● ●
●
●
●
●
●
●●
●
●
● ●
●● ●
● ●
●● ● ●
●
●●
●
●● ●
● ●
● ●
●
● ●
●
●
●
●
● ●
● ●
●
●
●
●
●●●
●
●
●
●● ●
● ●●
●
●
●
●●
● ●
●
●●
●
● ●
●●●
● ●
●
● ●
● ●
● ● ● ●
● ● ● ●
● ● ●●
●
● ● ●
● ● ●
●
●
●
●
●
●
● ●
● ● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●● ● ●
●●
● ●
● ●
● ●
●
● ●
● ●
● ●
●
●●●
●
●
● ●
●
●
●
●
●
●
● ●
●
● ● ● ● ●
●
●
●●
●● ● ●
●
●
●
●
●
● ● ● ● ●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
● ●
● ● ●
●
● ● ●
●● ●
●
●● ● ●
●
●
● ● ●
● ● ●
● ● ● ●
●● ●
●●
● ●●●●●
●
● ● ●
●
● ● ● ● ● ●
●
● ●
●
●
●
● ●
● ● ●
●
●
●
● ●●● ● ●●
●●
●
●● ●
●● ●
●
● ●●
● ●
●●●
●●●
●●
● ●
●
●●
●
●
●● ●●●
● ● ●
●
● ●●
● ● ● ●●
●
● ●●
●●●
● ●
●
●
●●
● ●●●
●●
●●
●
● ●
●●●●
●
●
●● ●
●
● ● ● ●
●
●
●
● ●●
●
●●
● ●
●
●
● ● ● ●
●
●
● ●
●
●● ●●
●●●
● ●●
●
●
● ●
●
●
● ●
●
● ●
●
● ● ●
●
●
●
● ●
● ● ● ●● ●
● ● ●
● ●●●●● ● ●
● ● ● ●
● ● ●● ●
●
●
●
●
●
●
● ● ●●●
●
●● ●
●● ● ● ● ●
●
●
●
●
●
● ●● ● ● ●●●● ● ●
●
●
●●●
●
● ●
●
●●●
● ●
●
●
● ● ●
●
●
●
●●
● ●
●
●
●● ● ●
● ●●
● ●● ● ●
● ●
●
● ● ●●●●
●●● ●●●
● ● ●●
●
●●
●● ●● ●
● ●
●●
● ●●●●
● ● ●● ●
● ●●●●
●
● ●●●●●●
●
●●
●● ●
●●●●
●●●● ●●●
●
● ●● ●● ● ● ●●
● ●● ●
●● ●●
● ●●
● ● ●●
●● ●●
●●●●
●●
● ● ●●●● ●
● ● ●
●●
● ●
● ●
● ●
● ●● ● ● ● ● ●
●
●
●
●●
● ●● ● ●● ●● ●
● ●●●● ● ●
● ● ● ●
● ●● ● ●● ●
● ●● ● ● ● ● ●
●
●
● ● ● ●
●
●●
●
● ●
●
●
●
●
●
●
● ●
●
● ●
●● ● ● ● ●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●●●
●
●
●
●
●
●
●
●
● ●
●
●●● ●
● ●●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
● ●
● ●
●
●
●
●
●
●
●
●
●
●
● ●
●● ● ●
●
●
●
●
● ●
●
●
●
●
●
● ●
●●
●● ●●
●
●
● ●● ●● ● ●●●
●●● ●
● ● ●
●●
●●
● ● ●
● ●●●●
●
●
●●●
● ●
●
●
●
●●
● ● ●
● ●
● ●●
●
●● ●
●
●
● ●
●●
●
●●●
●
●
● ● ●
●
●
●
●
●
●
●
●
●●●
●
●
●●
● ● ●
●
●●
●
●● ● ●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●●●
● ● ●
● ● ●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ● ● ● ●
●
●
●
●● ●
●
● ● ●
● ●● ●
●●
●
● ●
●● ●
● ●
●
●
●
●
●
●
●● ● ●●
●
●●
●
● ●
● ● ●
●
●
●
●
●
● ● ● ● ●● ●
● ●●
● ●
●
●
●●●
●
●●
●
●●
● ● ● ●
●
●
●
●
●
● ● ●
● ●
●
●
●
● ● ● ●
●
● ●
● ● ●
●
●● ●●●
● ●●●
●
●
●●●●●● ●
● ● ●● ●
●
● ● ●● ●● ●
● ●
● ●
● ●
●●
● ●
● ● ●
● ●●●● ● ●● ● ●●●
●
●●●● ●●
●
● ●● ●●
●
●● ● ●
●●●
●
●● ● ●
●
●● ●
● ● ●
●
●
●●●● ●●
●● ●
●
●●
● ● ●
● ● ●●
● ●●
●● ● ●●
● ● ● ● ● ●
● ●●●●● ●
● ●
●
● ● ● ●●● ●●
●
● ●●
●● ●
●
●
● ● ● ●
● ●●● ●
●
● ● ●
● ●
● ●● ●
● ●● ●● ● ● ● ●
●
● ● ● ●
●
●
●● ●●
●●
● ● ●
● ●
●
●
●
● ●
●
●
●
●
●●
● ● ● ●
●●
●
●●
●
●
●
●
● ●● ●
●● ●
● ●●●●
●
● ●●
●●●
● ●●● ●
● ●
●
●
● ● ●
● ●●
●●
● ●
●
●
● ● ●●
●
● ● ● ●
●
●
● ●●
●
●
● ● ● ●● ●
●
● ● ●
● ● ●
●
●
● ●●
●
●●
●●
● ●● ●●
● ● ●
● ● ●●
●●
●● ●
● ● ●●
●●
●●●●●●
●●
●●●●
● ●
●
●
●
● ●
●●● ● ● ●●● ●
●
●
● ●
● ●
● ●●
● ● ●
●● ● ●●
●●
●
●
●●
● ● ●●●●
● ●
● ●
● ● ● ●
● ● ●
●
● ●●
●● ●● ● ● ●
● ● ●
●
●
●
●●
● ●● ● ●
●●●●
●
● ● ●●
●● ●
● ● ● ● ●
●
●
●
●
●
● ●
●●● ● ●
●
●
● ●
●
●
●
● ●
●
●
● ●
●
● ●
●
●
● ● ●
● ●
●
●
● ●
●
● ●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●● ●
● ● ●
●●
●● ● ●
●
●
●
●
● ●
●
●●
●● ●●
●
●
●
●
●
●
●
● ●
● ● ● ●
● ●
● ●
●
●● ●
● ● ●● ● ●
●●● ●●●●●● ●
●
● ●● ● ●●
● ●●●● ●● ● ●●
●
●● ● ●● ●●
● ●●
● ● ●
● ●
●
● ●
●● ●●●
●●● ●●●
●
●
●
●●●●●
●
●
●
●
● ●
●● ●
● ●●● ●
● ●● ● ●
●● ●
● ●
● ●●●●
● ●● ●●
●
●
● ●
●
●
●● ● ● ● ●
● ●
● ●●
●
●
●
●
●●
●
● ● ● ● ●
●
●
●
●
●
●● ● ●● ●
●● ●
●
●
●
●●●
●
●
●
●
●
●
● ● ●
●
●
● ● ●
● ● ●
●
●
●
●
●
● ●
● ● ● ● ● ● ● ●●● ●● ●●●
● ●●● ● ● ●
●
● ● ● ●● ● ● ● ●● ● ● ● ● ●
● ●● ● ● ●●●● ● ●
● ●● ● ● ● ●
● ●● ● ● ●
● ● ●
●
●● ●● ● ● ●● ●● ●● ● ●● ● ●● ● ● ●● ● ●● ●●●● ● ●●●● ● ●● ●● ● ●● ●●● ● ●
●
● ●● ● ● ● ● ● ●
● ●●●● ● ●● ●● ●●● ● ● ● ● ●● ●
● ● ● ●● ● ●●● ● ●● ● ●● ● ●● ● ● ●
●●● ● ● ● ●●●
● ● ● ●
●●
●
●
●
● ● ● ●
●
● ● ● ●
●
●
●
●
●
●
● ●
●
●
●
●
● ● ●
●
●
●●●
●
●
●
●
●
● ● ● ●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
● ●● ● ● ●
●
●
●
● ● ●
●● ●
●
● ●●
●● ●
●
●
●
●
●
●
●●
● ●
●
● ● ●
●
●
●
●● ●
●
●
●
●
●
●
● ● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
● ● ● ●
●●●
●
●
●
●
●
●
●
● ● ●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●●
●
●
●
● ● ● ● ●
●●
●
●
●
●
● ● ●
●●
●
●
●
●●
●
●
● ●
● ●
●
●
● ● ●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ● ●
●
●
●
● ●
● ● ●
●
●
●
●
●
●
● ●
●
●
●
● ●
● ●
●
● ●
●
● ●● ●
●
●
●
●
●●
●
●
●●
●
● ●
●
●
●
●● ●
● ●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●●
●●
●
● ●● ●
●
●●
●
●
●
● ● ●
●
● ●● ● ●
● ●
●
● ● ●
●
●●
●
●
●
●
● ● ●
● ●
●
● ●●
●
●
●
●
●
● ● ●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
● ● ● ● ●●●
●
●
● ● ●
● ●
●
●
● ● ●
●● ●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
● ●●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
● ● ●
●
●
●
●●
●
● ● ●
●
● ● ●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
● ● ● ●● ●
●
●
●●
● ● ●
●
●
●
●
●
●
●
● ●
● ● ●
●
● ● ● ● ●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●●
● ● ● ● ● ● ●
●
●
●
●
●
●
● ●
●
●
●
●● ● ●
● ●●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●●
● ●
●
●●
●
●
●
●
●
●
●
●
●
● ●
● ●●
●
● wind ● ● ● ●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
● ●
● ●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ● ●
●
● ● ●
● ● ●
●
●
●
●
●
●
●
●● ●
● ● ●
●
●
●
● ●
●● ●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
● ● ●
●
●
●
●
● ● ●
●
●
● ●
● ●
●
●
●
●
● ● ●
●
●
●
●
●
● ●
● ● ●
●
●
●
●
●
●
● ● ●
●●
●●
●
●
●●
● ● ●
●●
● ● ●
● ●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
● ●
●
●
●● ● ●
● ●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●●
●
●
● ●
●
●
● ●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●●
●
●
●
●
●
● ●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ● ● ● ●●
●
●
● ●
●
●
●
● ● ● ●
● ●● ● ●
● ●●●
●
●
●
●
●
●
●
●
●
●
● ● ●
●●●● ●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●● ●
●
● ●
●
●
● ● ●
● ●
● ● ● ● ●
●
●
●
●
●
●
●
●
●
●
● ●
● ●● ●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ● ●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ● ● ● ●
●
●
●
●
●
●
● ● ●
●
● ●
●
●
●● ●
●
●
●
● ● ●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
● ●● ●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●●
● ● ●
●●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
● ● ● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
● ●
● ●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
● ●
● ●
●
● ● ●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
● ●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ● ● ● ●
●
●
●
●
●
●
●
●
●
● ●
● ●● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
● ●
●
● ●
●
●
●● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
● ● ● ●
●
●
●
●
●
●● ● ●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
● ●
●
●
●
●●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ● ●● ●
●
●
●
● ●●
● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●●
●
●
●
●
● ● ● ●
●
●●
●
● ● ●
●
● ● ●
●● ●
●
● ● ● ●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●●
●●
●
●
●
●
●
● ● ●
●
●
●● ● ●●
●
●
●
●
●
●
●
●
●
● ●●●
● ● ● ● ● ● ● ●
●●
●
●
●●
●
●
● ● ●
●
●
● ●
● ● ●
●
● ● ●
●
● ●
● ●
●
●
●
●
●
● ●
●● ●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●
● ●
●
●
● ● ●
● ● ●
●
●
●
●
●
●
● ●
●
●
● ● ●
●
●
●
● ●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●
● ● ●
●
●
● ●
● ● ● ●
●
●
● ● ●
●
●●
●
●
●
●● ●
●
●
●
●
●
●
● ● ● ● ● ●● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●●● ●
●
● ● ● ● ●
●
●
● ●
●
●
● ● ●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
● ● ● ●
● ● ●
●
●
●●
●
● ●
● ● ●
●
●
●● ●
●
● ●● ●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
● ● ●
●●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
● ●
● ● ● ●
● ●
● ● ●
●
●
●
●
●
● ●● ● ●
● ● ●●
●
●
●
●
●
●
●
●●
● ● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
● ● ●
●
●
●
●
●
●
●
●●
●
●● ●● ●
●
●
●
●
●
●
●
●
●
● ●● ●
● ●
●
●
●
●
●
● ●
● ●
●
●
● ●●
●
●●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
● ●
● ●
●
●●
●
● ● ● ● ● ●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●● ●
●
●
●
●● ● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
0 4 8
● ● ● ●
●
●
●
●
● ● ● ● ● ●● ● ●
●
●
● ● ●
●
●
● ● ●
●
● ● ●
● ● ●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
● ● ●
● ●●● ● ● ●● ●
●
● ● ●
● ● ● ● ● ●● ● ●
● ● ●
●
●
●
● ●● ●
●
●
●
● ●●
●
●
●
●
●
●
● ● ●
●
●
● ● ●
● ● ●
●
●
●
●
●
●
● ●
●
● ●● ●
●
●
●
●
● ● ●
●
●
● ● ●● ●
●
●
● ● ●
● ●
●
●
●
● ●● ●
●
●
●
●
● ● ● ● ● ● ●
●
●
●
● ● ●
● ●● ●
●
●
●
● ●●
●
●
● ● ● ● ●
●
●
● ● ● ●● ●
●
●
● ●
●
●
● ● ● ● ● ● ● ●
●
●
●
● ● ● ● ● ● ●
●
●
●
●
● ● ● ● ● ● ●
● ● ●
● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
● ● ●
●
● ● ●
●
●
●
●
●
●
●
●
●
● ●● ●
20 60
● ● ●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
● ● ●
● ●●
●
●
●
●
●
●●
●
●
●
● ●● ●
●
● ●● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
● ●
●
●●
●
● ●
●●
● ● ●
●●
●
●
● ●
●
●●
● ● ●
●
● ● ● ●
●
●●
● ● ●
●
●
●
● ● ●● ● ●
●
●
●
●
●
●
● ●
● ●
●
●
●
● ● ● ● ●
●● ●
●
●●
●
●
●
●
● ● ● ●
●
●●
●
● ●
● ● ●
● ●
●
●
● ●● ●
●
●
● ●
●● ●● ●
● ● ●● ●
● ●
● ●
●
●
●
●
● ●●
●
●
●
●
● ● ● ●
● ● ●
● ● ● ●● ●●
● ● ●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ● ● ●
●
●
●
● ● ● ●●
●
●●
●● ●
●
●
●
●
●
● ●
●
●●
●
● ●
●
● ● ●●
●
●●
● ●
●
●
●●
●
●
● ●
● ● ● ●
● ●
● ●
●
●
●
●
●
●●
●
●
●
●●●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●●
●
●
●● ● ●
● ● ● ●● ●
●
●
●
●
● ● ●
●●
●
●
●
●
● ●● ●
●●
● ●●● ●
●
●
●
●
●
●
●
●
● ●●● ●
●
● ● ●
●
●
●
●
● ● ●
●●
●
● ●
●●
●
● ● ●
● ● ●
●
●
● ● ●
●
●
● ● ●
●
● ● ● ●
●
●●
● ● ●
●
●
●
● ● ● ● ● ●
●
● ●
● ●
● ● ● ●
●
● ● ● ● ●
●
● ● ●
●●
●
●
● ● ● ● ● ● ●●
●
●
●
● ● ●
● ● ●
●
● ●● ●
●
●●● ● ●
●
●
● ●●●
●
● ● ● ●
●
● ●●●● ●
●
● ● ●
●
●
●
● ●●● ● ●
● ●● ●●
●
●
●
●
●
●●● ● ●●
● ● ●
●
●
●
● ●● ● ● ● ●
●● ● ●
● ●
●
● ●
●
●
● ● ●
●
● ● ●
●
●
● ●
●●
●
●
●
●
●
●
●●
●
●
●
●● ●●
●
● ●●●
●
●
●
●
●
●
● ● ●●
● ●● ●●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ● ● ●
●
●
● ●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●●
● ● ●● ●
●
●
●
● ●
●
●
●
●
●
● ● ●
● ● ●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
● ●
● ●
●
●
●
●
● ● ●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
● ● ● ●
●
●
●
●● ● ● ●
●
●
● ●
●
●
●
● ●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ● ●
●
●
●
● ● ●
● ● ● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
● ●
●
●
● ●
● ●
● ● ●
● ●
●
● ●
●
●● ● ●
●
●
●
● ●
● ●
●
●
●
●
●
●
●
● ●
●●● ●
●
●
●●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
humidity
● ● ●
●
● ●
●
●
●
●
●
● ●
●
●
● ● ●
● ● ●●●
●
●
●
●
● ●●
●●
●
●
●
● ●● ●
●●●
● ●
● ●
●
●
●
●
● ●
●
●
●
● ●●● ●
●
● ● ●
●
●
●●
●
●
●
●
●
●
● ●●
●
●
● ● ●
●
●
●
●
●
● ●
●
● ●
● ● ● ●
● ● ●
●
● ●
● ● ●
●
●
●
● ● ● ● ●●
●
●
●●
● ●
● ● ● ●
●
●
●
● ● ●
●
● ●●●● ●
●
● ● ● ● ●
● ● ●● ● ●
● ● ●
●
●
●
● ●● ●
●
●
● ●
●
● ●
● ● ● ●
●
●
●● ●● ●
●
●●
●● ● ●● ●
● ● ●● ● ● ●● ● ●
● ●● ● ● ●
● ● ● ●
●
●
●
●
● ●●
●● ● ● ●
●
●
● ●● ●
●
●
●● ●●● ●
●
● ● ● ●
● ●
● ●
●
●●
● ● ●
●
● ● ●
●
●
●
●
●
● ●
●
●
●
● ● ●
●● ● ●
●
●●● ● ●
●
●
●
●
● ●●●
● ●● ●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
● ●
●
● ●
●●
●
●
●
● ●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ● ●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ● ●● ● ●
●
●
●
● ● ●● ● ● ●
●
●
●
●
●
●
● ● ●
●
●
● ● ●
● ●
●
● ●
● ● ●
●
● ● ● ●
●
●
●
●
●● ● ● ● ●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
● ● ●
●
● ●
● ●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●
● ●
●● ●
●
●●
●
●
●
● ● ● ● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●●
● ●
●
●
● ●
● ●
●
● ● ●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●● ●●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
● ● ●●●●
●
● ● ●
●
●
●●
●
●
●
●
●
●
●
● ● ● ●
●
● ●
●●
● ● ●
●
● ●
●
●
●
● ● ● ●
●
● ●
● ●
●
● ● ● ● ●
● ●●
●
● ●
●
●
●●● ● ● ●
● ● ● ●●
●
●
●
●
●● ● ●
● ●
●
●
● ●
● ●
●
● ●● ●
●
●
●
●
●●
●
●
●●
●
●
●
● ● ● ● ●● ● ● ●
●
●
● ●● ●● ●●● ●
● ● ● ●
●
●
●
● ● ● ●
● ● ●
● ●●
● ● ●
●
● ● ●
● ●
●
●
●
●
●
●●●
● ● ● ●● ●
●
●
●●
●●
●
●
● ●
●● ● ●
●● ● ●
●
●
● ● ●
●
●
●●
●
● ● ●●
● ● ●● ●
●
●
● ● ● ●● ● ●●
●
●●●
●● ●
●
●
●
●
●
● ●●●
● ●
● ●● ●
● ● ●
●
●
●
● ● ●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ● ●
● ● ● ●
●
●
●
● ●
●●
●
●
●
●
●● ●● ●
●●
●
●●● ●
●
●
●●
●
●
●
● ●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●●
●
● ●
●
●
●●
● ● ●
● ● ●
●
●
● ●
●
●●
● ● ●
●
● ● ● ●
●
●
● ● ●
●
●
● ● ● ● ● ● ●
●
●
● ●
● ●
● ● ● ●
●
●
●
● ● ●
●
●
●
●●
●
●
● ● ● ● ●
● ●●
●
● ● ● ●
●
● ● ●
●
●●
● ●● ●
●
● ● ● ●
● ● ● ●●●
●
● ●● ●
● ●
●
●●
● ● ●●
● ●
●
●
●●
● ● ● ●●●● ● ●
● ●
● ●
●
●
●
●
●
●
● ●
●●●● ●
●
●
●
●
●
●
● ● ● ● ●● ●
●
●
●● ●
●
● ● ●
● ● ●
●
● ● ● ●
●
● ● ●
●
●
●
●
●
●
●●
●
●
●●
● ●
● ●● ●
●
●
●● ●●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ● ●
●
● ●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ● ●
●
●
●
●
●
●
●
● ●
● ● ●
●
●
●
●
●
● ● ●
●● ●● ● ● ●
●
●
●
● ●●●● ● ●
●
● ●●● ● ●
●
● ●● ● ● ●
●
●
●
●
●● ● ●● ● ●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
●
● ●
●● ●● ● ● ● ●
● ●
● ● ● ●●●●
●●
●
●
●
● ● ● ●●
●
●
●
●
●
● ●
● ● ● ●
● ● ● ● ● ● ●
●
●
●
● ●
● ● ●
●●
● ● ● ● ● ●
●
●
●
●
●
● ●●
●
●
●
●● ● ● ● ●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
● ●●
●
●
●
●● ●
●
●
●
●
●
●
● ● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
● ● ●
●
●
● ● ●
●
●
● ● ● ● ●
●●
●
● ● ●
● ●
●
●
●
●
●
●
● ●●● ● ● ●
●
●
●
● ●●
●
●
● ● ●
●
●
●
●
● ● ●
●
●
●
●
●
●●
●
●
● ● ●
●
● ● ● ● ● ● ● ● ● ●
● ● ●
● ● ● ● ● ●● ●
●
●● ● ● ●● ●● ● ● ●
● ●● ● ● ●
● ● ● ●● ● ● ●
● ● ●
●
●
●
● ● ●
●
● ● ● ●● ● ●●
● ●● ● ●●● ● ●● ●
● ● ●
● ● ● ●●● ● ● ●● ● ● ●
●
●
●
● ● ●● ● ● ● ● ● ●
●
● ● ● ● ●
● ● ● ● ●
● ● ●
● ● ●
●
●
●
●
●
●
● ● ●
●
●
●
● ● ● ● ●●● ● ● ●
●
●
●
●
● ●
● ●
●
● ●
●
●
●
● ●● ● ●● ● ●
● ●● ●●●
●
●
●●
●
●●
●●
●
● ●
● ● ● ●●●● ● ● ● ●
●●
● ● ● ● ●
●
●
● ●● ● ●● ●
● ● ●●●
●
●
●
●
●
● ● ●
●
●
●●
● ●
● ●
●
●
●
●
●
●
●
●
●
●
● ● ● ● ● ●
●
● ● ● ●
●● ●
● ● ● ● ●
●
● ● ●● ● ●
● ●
● ●
●
●
● ●
● ●
●
●
● ● ●●●
● ● ●
●●
● ●
●
●
● ●●
●
●
● ●
● ●
●
● ● ●
●
●
●
●
● ●●● ● ●
●
●
●● ● ●
● ● ● ●
●●
●
●
●
●
● ● ●
●
● ●
● ●
●● ●
● ●●
● ●●● ●
●
●●● ●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
●
●● ● ●
● ● ● ● ● ● ●
● ●
● ●
●
●
●
●
●
●
●
●●
● ●
●● ●
●● ●●●
●
●
● ●
●
●
●
● ●● ●
●● ● ●
●
● ● ● ● ●
●
●
● ●● ●
●
●●
●●●●●
●●
● ● ● ●
●●
●
● ●● ● ●●
● ●
●● ●●●●
●●● ● ●●
●● ●●
●
●
●
●
● ● ●
● ● ●●●
●
● ● ● ●
● ●●●
● ● ●
●
●
●
●
● ●●
●
● ● ● ● ● ●●
●
●
●
● ● ●
●
●
●●
●● ●
● ●
● ●●
●
●
●
● ●
●
●
●● ●
●
● ● ●●
●
● ●● ●
●● ●
● ●
● ●
●
● ●●●●● ●● ●
● ●●
● ● ● ●
●
●●● ●
● ●
● ●
●●
● ●●●● ● ● ●
●
●
●
●
●
● ●●● ● ● ● ●●● ● ● ●● ●●● ●
●
●● ● ●● ●
● ●
●● ●
●●
● ● ● ● ●
●
●
●
● ●
● ●●
●
●
●
●
● ●●● ●●●●
● ● ●
●●● ● ●●●
● ●
●
●●
●
●
● ● ● ●●●
●● ●
●●
●
●
●
● ● ● ●●
●
●●
●
●● ●●●
● ●
● ● ●
●
● ●●●● ●
●
● ● ●● ● ●
● ●●
●
●
●
● ●● ●
●
●
● ● ●
●
●
●
● ●
●
● ● ●
●
● ●●
●
● ● ●
●
●
●● ●
●
●
●
● ●
●● ● ●
●
●
●
● ● ●
● ● ●
●
●
●
●
●
●
●
●
●
● ●
●● ●
●
●
●
● ●
●
●
● ●
●●
●
●
●
●●●
●
●
●
●
● ●
● ● ●
●
●
●
● ● ●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●●
●
●
●
●
●
●
●
●● ●
●
●● ●
●
●
●
●
● ●
● ●
●
●
●
●●
●
●
●
● ●
●● ●
●
●
●
●
●
●
● ●
●
●
● ●
●● ●
● ●
● ● ● ●
●
● ●
●
● ● ●
●
●
● ●
● ●●
●
●
●
●
● ●
● ●
●
●
●
●
● ●●
● ●
●
●●
●
● ●●
●
●
●
●
●● ●
●
●
●
● ●
●●●
● ●
●
● ●
●●
●● ● ●
● ● ●
● ●●●
●
● ● ●
● ● ● ●
●
● ●
●
●
●
● ●
●●
●
●
●
●●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
●●
●
● ● ● ●
● ●
●
●
● ●
●
●
●
● ● ● ●
●●
●
●●●
●
●
● ●
● ●●●
● ●
●
● ● ● ●
●●
●
●
●
●
●● ● ●
●
●
●
●
●
● ● ● ●
●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
● ● ●
● ●
●
●
●
●
● ● ●
●
● ● ● ●
●
●
●● ●
● ● ●
●
● ●
●
●● ●
● ●
● ●●●●●
●
● ●●
●
● ● ●
● ● ●
●
● ●
●
●
●
● ●
● ●
●
●
●
●● ●● ● ● ●
●●
●
● ● ●
●● ●
●
● ●●
● ●
● ●●
●● ●
●●
● ● ●
●
●●● ●
●
●
●● ● ●●●
●
● ●
●
● ●●
● ● ●●● ●
●●
●
●● ●
● ● ●
● ●
●
●
●●
●
●
●● ● ●
●●
●
●
●
● ●
●
●● ●
●
●
● ●●
●
● ● ●
●
●
● ●
● ● ●
●
●●
●
●
● ● ●
●
●
● ●
●
● ●●
● ●●
●● ●
●
●
● ●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
temp
●
● ● ● ●
●
● ●
●
●●
●
● ● ●
●
●
●
●
● ● ●
●●●
●
●
●
●
●
● ● ●●
●
● ●
●
●
●
●
●
●
●●●
●
●●● ● ●●
●
●
●
● ● ● ● ●
●
●
● ● ●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●●
●
●
● ● ●
●
●
●
●
●●
● ● ● ●
●
●
●
●
● ●
●●
●
●
●
●
● ●
● ●
●● ●●
●
●
●
●
● ●● ● ●
●
●●●
● ● ●
● ● ●
●● ●
● ●
●● ●
●
●● ●
●
●
●
●● ●●●
●
●
●
● ●
● ● ●
● ● ●
● ● ●
● ● ● ●
● ●
●
● ●
● ●
●
●
●
● ●
●
●
● ●
●
●
●
● ● ●
●
● ●
●
●
● ●
● ●
●
●
●
●
● ● ●
●
●
●
●●
●
●
●
● ●
● ● ● ●●
●● ● ●
● ● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ● ● ●
●
●
●
● ● ● ● ●
● ●
●
●
● ●
●●
● ● ●
● ●
●
● ●
● ●
●●●
●
●
●●
●
●
●
●
● ● ● ●●
●
● ●●
● ●
● ● ●
●
●
●
●
● ● ● ● ●●●
● ● ●
● ●
●
●
● ●●
●
●
●
●
●●
● ● ● ●
●●
●
●
●
● ●●●
●
●
●
●
●
●● ● ●
●●
●
● ●
●
●● ●● ●
● ●●●
●
● ●
● ●● ● ● ●
● ● ●●
●
● ●● ● ●● ●
● ●
● ●
●●
● ●
● ●
●●●
●
● ● ●●● ●
●
●
●●●● ●●●● ● ●● ●
●
●● ● ●
●● ●
● ● ● ● ●
●●
● ● ●
● ●● ●
●
● ●
●
● ● ●
● ●
●
●
●
●
●
● ●
● ● ● ●● ●
●
●●
●● ● ●
●●● ●
●● ●
● ● ● ● ● ●
● ● ●●●
●●●● ●●● ● ●
●
● ●● ●● ● ●
●
●● ●●
●● ●
●
●
● ● ●
● ●●● ●
●● ● ● ●
●● ● ●●
●
●
●
● ●● ● ● ●
●
● ● ●
●
●
●● ● ●
●
●
● ● ●
● ●
●
●
●
● ●
●●
●● ●●●
●● ● ● ●
● ●●
●
● ●
● ●
●
●
● ●● ●
●● ●
● ● ● ●●
●
●
●
● ●●
●
●
●
●●
● ●
● ●
●
●
●
●
●
● ●
●
●
●
●
● ● ● ● ●
●
● ● ● ●
●
●
● ●●
● ●
●
● ●●●● ●
● ● ●
● ●
● ● ● ●
●
● ● ●
●●
●●
●●
● ●● ●●
●● ● ●
●
●
● ●
● ●● ●
●
●●
● ●●●●
● ●
●● ● ● ● ● ● ●● ●
● ●
●●●
●●
●
● ●
●●
● ●●● ● ●● ● ●● ●
●
● ●
● ●
● ●
●
● ●
● ● ● ●
●●
●
●
●● ● ●● ● ●
● ●
●●
● ●●
● ●
●
●
● ● ●
●● ●●● ●
● ●
●
●
●
●● ● ●● ●
●●● ●
●
●● ●●
●●●
● ● ● ●
●
●
● ●
●
●
●●
●
● ● ● ●
●
●
●●
●
●
●
●
● ●
●
●
● ●
●
● ●
● ●
● ● ●
●
●
●
● ● ●
●
● ●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
● ● ●
●
●
●
●
●● ●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●●
● ● ●
● ●
●
●
●● ●
● ●
●
●● ●
●●
●● ●●●● ● ● ●
●
●
●●●● ●
●●● ●●● ● ●
●
●
● ● ● ●●●
●
●●
●
● ● ●
●
● ●
●
●● ●
● ● ● ●● ●
●● ●● ●
●● ●
●
●●●
● ● ●● ●
●
●
●
●
● ● ●
●● ●
●● ●● ● ● ●
●●
●
● ●
●● ●●
●●
● ●● ● ●
● ● ●
●
● ●
● ●
●
●
●●●● ● ●
● ●
● ● ● ●
●
●
●
●
●●
●● ● ● ●
●
●
●
●
●
●● ● ● ● ●
● ●●
●
●
●
●● ●
●
●
●
●
●
●● ● ●
●
●
● ● ●
● ● ●
●
●
●
●
●●
●
30 70
● ●● ● ●● ●●
●
●
● ●● ● ● ● ●●●●●
● ●● ● ●● ●● ●● ● ●●●●●● ●
●
● ●● ●
●
●
●
●
● ● ●
●
●
●
● ●●●● ● ●● ●
● ● ●
●
● ● ●
●
● ●●● ● ●● ●
●
●
● ● ●
● ●●● ● ●
●
● ● ● ● ●● ●●● ●●
●
●
●
● ●●●● ● ● ●●●● ●
●
●●
● ● ●●
● ● ●● ●
● ● ● ● ● ● ●●● ● ●●
● ●●● ●
●
● ● ●●
●
0 3000
●
●
●
●
●
● ●
●
●
●
●
●
●●
● ●
●
● ● ●
● ●● ●
● ●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●● ●
●
●
●●
●
●●
●●●
●
●
●
● ●● ●●
●●
●●
●
●
● ●
●●
●
●●
●
●●
●
●●
● ● ●
●
● ● ● ●
●
●●
● ● ●
●
●
●
●
●
●
● ● ●
● ●
● ●
●
●
● ●
● ●
●
●
●
●
● ● ●
●
● ●
● ●
●●
●
●
●
●
●● ●
●
●●
●
● ●
●● ●
●
●
●
●
●
●
● ● ●● ●●
●
●●●
● ●●
● ● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
● ●
● ●●●● ●
● ● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ● ●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
● ● ●
●
●
● ●
●
●●●
●● ●
●
● ●●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ● ●
●
●
● ●
●●
●
●
● ● ●
● ●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●●●
●
●●
● ● ●●
●
●● ● ●● ●
●●
●
●
● ●
●
● ●●
●
●●
●
●
●
●
●
●
●
● ●●
●
● ● ● ●
●
●
● ●
●
●
●
●
●
● ●●● ● ● ● ●
● ●
●
● ● ●
●
●
● ● ●
●
●
● ●
● ●
●● ●
●● ●● ● ●●
●●● ● ●
●
● ● ● ●● ●
●●●
●
●
●●
●
●
●
● ● ●
●●
●
●
● ● ●
●
●●
●
●
● ●
● ●●● ●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●● ● ●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●
●
● ●
●
●●
●
●
●
●● ●●●
●
● ● ●
● ●●
●
●●
● ● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ● ● ●
●
●
●
● ●
●
●●
●
●
● ● ●
●
● ● ●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
● ●
● ●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●● ●
●
●
●
●
●
● ● ●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
● ● ●
●
●
● ● ●
●
●
●
●
● ●
● ●
●
●
●
●● ●
●
●
● ●
● ●
●●
●
● ●
●
● ●
●
●
● ●
●
● ●
● ● ●
●
●
●● ● ●
●
●
●
●
● ● ●
● ●
●
● ●
●
●●
●
●
●
●
● ●
● ●
●
●
●
●
● ● ●
● ●
●
●● ● ●
●
●
●● ●
●
●●
●
●
● ●
●
●
●
●
● ●
●
●●
● ●
●
●
●
●
●●
●
● ●
●
● ●
●
●
● ●●
●
●
● ● ●
● ●
●
●
●
● ●
●
●
●
● ● ●
● ● ● ●
●
● ●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
● ●
●●
● ●
●
●
●
● ●
●
●
●
● ●
●
●
●
●
● ●
●
● ● ●
●
●
●
●
●●
● ● ●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
● ●
●
●
●
●
●
●● ●
● ● ●
●
●
●
●
●●
●
●
●
●
●
●
●● ● ●
●
● ●●
●
●● ● ● ●
●
●
● ●
●
●
● ●
● ●
●●
●
●
●
●● ●● ●●●
●●
●● ●●
●● ●
●
●
●
● ● ●
● ●
● ● ●
●
● ●
● ●
●
●
●
●
●
●● ●
●
●●
●
●
●●●
●
●
●●
●
● ●
●
●● ● ●●
● ●
● ●
●●
●
●
●
●
●
●● ●
●
●
●
●
● ●
●●
●
●
●
● ●
●
●
●● ●
●
●
● ● ●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●●
●
●
● ● ●
● ●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●●●
●●
●
●
●
● ●
● ●●●
●
●●
● ● ● ●●
●
●
●●●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●●●
●
●
● ●
● ● ●
● ● ●
● ● ● ● ●
●
●
●
●
●
● ●● ●
● ●
●
●
● ●
● ● ●●
●
●
●
●
●
●
●●
●
●●
●● ●
● ●●
●●●●
● ● ● ●●●
●
●
●
● ● ●● ●● ●
●
●
● ●
● ● ●
●
●
●
●● ●●
●
●
●
●
●
●
● ●
●
●●
●
●● ●●
● ●●
●
●
●●
●
●
●
●
●● ●
●
●
●
● ●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ● ● ● ●
●
●
●
● ●●●●
●
●
●●
● ● ●
●
●● ● ●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
ibh
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
● ●
●
●
●
●
●● ●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●●
●●
●●
● ●
●
●
●
●● ●
●
●●●
● ● ●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
● ●
●
●
●
●
●
●
●
●● ● ●
●
● ●
●
●
●● ●
●
●
●
●
●
● ●
●
● ●
● ● ●● ● ● ● ●
● ●
●
●
●
●
●
●
●●
● ● ●
●
●● ●
●● ●● ● ●
● ● ●●
●● ● ●●●●●
●
●●
●
● ● ●
●
● ● ● ● ●
●
●
●
● ●● ●
●●
●
●
● ● ● ●
● ●
●
●●
● ●●
●●●
●
●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●●
●
●
●
● ●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
● ●
●
●●
●
●●● ●
●●
●
● ● ●●
●● ● ●● ●
● ●
●
●
●
●
●
●●
●
●
●
●●●
●
● ●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
● ●
●
● ●
●
●
●●
●
●
●
●●
●
● ● ●
●
●
●●
●
●● ●●
●
●
●●●
●
●
● ● ●●●
●
●
● ●●
●
●
●
●
●
●
● ●
●
●●
●
● ●
●
●
●
●●
● ●●
●
● ● ● ●
●
●
● ●
●
●
●
●
●
● ●● ● ●
●
●
● ●
● ●
● ● ●
●
●
● ● ●
●
●●
●
●●
● ●
●● ●●
●●●
● ● ●●
●● ●
● ●
●
● ● ● ●●
●
●●
● ●● ● ● ●
●
●
● ●●
●●
●
●
●
● ●
●
●
●
●
●●
● ●● ●● ●
●
●
● ●
●
●
●
●●
●
● ● ●
●
●
●
●
●
●
●●
●
●
●●
●
●● ● ●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●●
● ● ● ●
●
●
●
● ● ●●
●
●
● ● ●
●
●
●●
● ●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
● ● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●●●
●
●
●
● ●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●●●
●●
●
● ●
●
●●
●
●
●
●
●
● ● ●
●
●
●
● ●● ●● ● ● ●
●
●
●
●
●
●
●
●
● ● ● ● ●
●
●
●●
● ●● ●●●
●●●
● ●
●
● ●
●●
● ● ●
●●
●
● ●●
●
●
●
● ●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
● ●
●
● ●
●
●
●
●
● ●
●●
●
●●
●
●
●
●
●●
● ● ●
●
●
● ●●
●
●
● ●●
●
●
●
● ●● ●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●●● ● ●
●
●
●
●
● ●●
●
● ●●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
● ● ●
●
●
●
●
● ● ● ● ●
● ● ●
●
●
●
●
●
●
● ● ●
●
●
●
● ●● ●
●
●
●
●
●
● ●● ●
●
●
●
● ● ●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
● ●● ● ● ● ●● ● ● ● ● ●
●
●
● ● ●
●
●
●
●
●
●●
●
●
● ●● ●
●
● ●●●
●
●
● ●●
●
●
●
● ● ●
●
●
●
● ●● ●
● ●●
● ● ●
●
● ● ●
●
●
● ●● ●
●
●
●
●
● ● ●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
● ● ● ● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ● ●
● ●
● ● ● ● ●
●
● ● ●
●
●
●
●● ●
● ●
● ●
●
● ● ●
●
● ● ●
●● ●● ●
●
●●
● ●● ● ●
●
●●
●
● ●●●
●●
● ●● ●
●
●
●
●
●
● ● ●
●
●
●●
● ● ●
●●
●
●
● ●●
●
●
●●●
●●
●
● ●● ● ●
●
●
● ● ●
●
●●
●● ●
●
●
●
●● ●● ● ●
●
● ● ●
●●
● ●
● ●
●
●
● ● ● ● ● ●
●●
●
●
●
● ● ●
●
● ●
●
●
●●
● ●
●
●●
● ● ●●
●
● ●●● ● ●
● ●
●● ●●●
● ●●
●●
●
●
●
●●
●●
●
●
●
●
●
● ●● ●
● ●●
● ● ●●● ●
●
●
●●
●
●
●
● ●
●
●
● ● ●
●
● ●
●
●
●
●
●
●● ● ●
●● ● ● ●●
●● ●
●
●
●
●
●
●
●
●
● ●
● ●
●
● ●
●
●●
● ●●
●●
●
● ●
●●
●
●
●● ●
● ● ● ●
●
●● ●
●
●
●
●
●
●
●● ●
● ● ●●● ●● ●
● ● ●
●
●●
● ● ●
●
●
●●
●
●
● ●
●
● ●● ●
●
●● ● ●
●●
●
●
● ● ●
●
● ●
●
● ● ●
●
●
●
●
●
●
● ●● ●
● ● ●
●
●● ●
●
●
●
●
● ● ●
●
● ●● ● ●● ● ●
●
●
●
● ● ●
●
●
●●● ● ●
●●
●
● ● ● ●
●
●
●●
●●
●
●
●
● ●
●
●
●● ●
●
●● ●● ●
●
●
●
● ●● ● ●
●●● ●● ●
● ● ●
●
●
●● ●● ●
●● ●● ●
●
●
● ●● ●
● ● ●● ●
●
●
●
● ● ●
●● ●● ●●●●●
●● ●● ● ●
●
●
●●
●
●●
● ●
●
●●
● ● ● ● ●● ● ●
●
●
●
● ●
● ●●
● ● ●
●
●● ● ●●●
● ●● ● ●●
●
●
●
● ● ●
● ●
●●
●●
● ●● ● ●● ●
●● ●
● ●
●
●●
●
●● ● ●●
●●●●●
●
●
● ●● ● ●
●
●
● ● ●
●
●
●● ● ●● ●
●
● ● ● ●● ●
● ●●
● ●
●●
● ● ●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
● ●
● ● ●
●
● ● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
● ● ●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
● ●● ●
●
● ●●
● ●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
● ● ●
●
●●
●
●
●
●
● ●●
●●●
●
● ●●
●
●
●
●
● ●
● ●
●
●
●●
●●
●
●
●
● ●
● ●
●●
●
●
●
●
●
●●
●
●
● ●
●● ●
● ●
● ● ● ●
●● ●
● ●
● ●●
● ●
● ●
● ●●
●
●
●
●
● ●
● ●
●
●
●
●
● ● ●
●
●
●
●●
●
● ●●
●
●
●
●
● ● ●
●
● ●
●
● ●
●●●
●
●
●
●
● ●
● ●
● ● ● ●
● ●● ●
● ●● ●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
● ●
● ● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ● ●
●
● ● ●● ● ●
●●
●
●
● ●
● ●
●
●
●
●
●
●
● ●
●
●
●● ●
●
●
● ●
●● ●● ●
●
● ●
●
● ●● ●
●
●
●
●●
●
●●
● ●
●
●
●
●
● ● ●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
● ● ●
●
● ●
●
●●
●
●
●
● ● ●
●
● ●
●
●
● ●
● ●
●
● ●
●● ● ●● ●
●
● ●●
●
● ● ●
●
●
● ●
● ●
●
●
●
●●
●● ●
●
●
●
● ● ● ● ● ●●
●●
●
●●
●
● ● ●
●
● ●●
● ●
● ● ●
● ●●
● ●
● ● ●
●
●●● ●
● ●
●●
●●
●
● ●
●
● ● ●
● ● ● ● ● ●●
●●
●● ●
● ● ●
● ●
●
●
●●
●
●●● ●
●●
●
●
●
● ●
● ● ●●
●
●
●
●
● ●
●
● ● ● ●
●
●
●
● ●●
●
●●
● ●
●
●● ● ● ●
● ●
● ●
●
● ●●
● ●●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
● ● ●
●
●
● ●
●
● ● ●●●
●
● ●
● ● ● ●
● ●● ●
●
●
●
●
●
●
●
●
● ●
●
●
●● ●●
●
●
● ● ●
● ●● ●
● ● ●
●
● ●
●
●●
●●
● ● ●● ●●● ● ●● ● ●
●
●
● ●●
●
● ●
●
●
●
● ●
●
●
● ● ●
●
●
●●●
● ●
●
● ●
●
●
●
●●
●
●
● ●● ●
●
●
●
● ●● ● ● ●
● ● ●●
● ● ●
●
●
● ● ● ● ●● ●
● ● ●
● ●● ●
● ●
●● ●● ●
●
●●
● ● ●
●
● ●●●● ● ●
●
●●●
●●● ●
● ● ● ●
●●●
●
●
●
●● ●● ● ● ● ●●
● ● ● ●
● ●● ●
● ●
● ●●●
● ● ●●
●
● ● ●●
●●
● ●●
●
●
●
● ● ●
●●
●
●
● ●
● ●
● ● ●
● ●● ● ● ● ●
● ●
● ●● ● ● ●● ●● ● ●
● ●●● ● ●● ●
●
●● ●
●
●● ● ●● ●
●● ●●
● ●●● ●
●
● ● ● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ● ● ● ●
●
●
●
●
●
●
●
●
● ●
● ● ●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
● ● ● ● ●
●
●
● ● ●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
● ● ● ●
●
●
● ● ●
●
●●
●●
●
●
●●
●●● ●●
●

lmobj glmobj require(mgcv)
> glmobj summary(glmobj)
Family: gaussian
Link function: identity
Formula:
log(O3) ~ vh + wind + humidity + temp + ibh + dpg + ibt + vis +
doy
13

Parametric coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 6.301e-01 2.684e+00 0.235 0.81453
vh -1.244e-05 4.908e-04 -0.025 0.97980
wind -8.403e-03 1.226e-02 -0.685 0.49357
humidity 5.084e-03 1.713e-03 2.968 0.00322 **
temp 2.993e-02 4.527e-03 6.611 1.60e-10 ***
ibh -8.495e-05 2.687e-05 -3.161 0.00172 **
dpg 5.470e-04 1.026e-03 0.533 0.59414
ibt 4.902e-04 1.237e-03 0.396 0.69221
vis -8.395e-04 3.410e-04 -2.462 0.01434 *
doy -1.021e-03 2.522e-04 -4.049 6.47e-05 ***
---
Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1
R-sq.(adj) = 0.709 Deviance explained = 71.7\%
GCV score = 0.16808 Scale est. = 0.16298 n = 330
14

If y is skew and positive, alternative 1
Take the logarithm of y, log(y) is more symmetric
log(y) = η + ε
y = exp (η + ε) = exp (η) · exp (ε)
E(y) = exp (η) · E(exp (ε))
E(exp (ε)) = exp (σ 2 /2) if ε ∼ N(0, σ 2 )
15

If y is skew and positive, alternative 2
E(y) = µ
log(µ) = η
y ∼ Gamma(µ, parameter2)
16

Ex: Ozone data - Gamma regression
>
> ### GLM, O3 ~ Gamma-distributed with identity link
> glmobj summary(glmobj)
Family: Gamma
Link function: log
Formula:
O3 ~ vh + wind + humidity + temp + ibh + dpg + ibt + vis + doy
17

Parametric coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 3.805e-01 2.583e+00 0.147 0.882957
vh 6.323e-05 4.723e-04 0.134 0.893591
wind -8.965e-03 1.180e-02 -0.760 0.447870
humidity 5.752e-03 1.648e-03 3.490 0.000551 ***
temp 2.382e-02 4.356e-03 5.469 9.15e-08 ***
ibh -6.420e-05 2.586e-05 -2.483 0.013548 *
dpg 9.455e-04 9.869e-04 0.958 0.338775
ibt 1.527e-03 1.191e-03 1.283 0.200561
vis -8.488e-04 3.281e-04 -2.587 0.010126 *
doy -1.011e-03 2.427e-04 -4.165 4.01e-05 ***
---
Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1
R-sq.(adj) = 0.736 Deviance explained = 71.1\%
GCV score = 0.15567 Scale est. = 0.15095 n = 330
18

Ex: Precipitation - binary logistic regression
• 10 years data with daily precipitation at Tryvann, Oslo
• Response is P01: 1 if precipitation, 0 if not, at day t
• 6 predictors
⋆ P01.L1 - 1 if precipitation, 0 if not, at day t − 1
⋆ P01.L2 - 1 if precipitation, 0 if not, at day t − 2
⋆ P01.L3 - 1 if precipitation, 0 if not, at day t − 3
⋆ Prep.L1 - amount of precipitation (mm) day t − 1
⋆ Prep.L2 - amount of precipitation (mm) day t − 2
⋆ Prep.L3 - amount of precipitation (mm) day t − 3
19

R code
### Note: Linux style on directory structure used here
> Tryvann.dat
> ### load the mgcv library
> require(mgcv)
>
> ### Logistic regression with binary precipitation (0=no, 1=yes) as r
> glmobj summary(glmobj)
Family: binomial
Link function: logit
Formula:
P01 ~ P01.L1 + P01.L2 + P01.L3 + Prep.L1 + Prep.L2 + Prep.L3
20

Parametric coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -0.774966 0.068825 -11.260 < 2e-16 ***
P01.L1 1.064480 0.083536 12.743 < 2e-16 ***
P01.L2 0.322184 0.085679 3.760 0.00017 ***
P01.L3 0.411833 0.084189 4.892 1.00e-06 ***
Prep.L1 0.032718 0.007428 4.405 1.06e-05 ***
Prep.L2 -0.009255 0.006212 -1.490 0.13626
Prep.L3 -0.004704 0.006028 -0.780 0.43523
---
Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1
R-sq.(adj) = 0.125 Deviance explained = 9.43\%
UBRE score = 0.24364 Scale est. = 1 n = 3420
21

Ex: Simulated data - Poisson regression
n x1 x2 x3
> eta mu
> y
> data.obj
> ### load the mgcv library
> require(mgcv)
Loading required package: mgcv
This is mgcv 1.7-6. For overview type ’help("mgcv-package")’.
>
22

### Estimation
> glmobj summary(glmobj)
Family: poisson
Link function: log
Formula:
y ~ x1 + x2 + x3
23

Parametric coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 0.154044 0.071537 2.153 0.0313 *
x1 0.802362 0.051533 15.570 < 2e-16 ***
x2 -0.038901 0.009412 -4.133 3.58e-05 ***
x3 -0.107352 0.059222 -1.813 0.0699 .
---
Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1
R-sq.(adj) = -0.00555 Deviance explained = 21.1\%
UBRE score = 3.6933 Scale est. = 1 n = 200
24

Prediction for new data points
> new.data.obj predict(glmobj,newdata=new.data.obj,type="response")
1 2
1.166542 2.602326
25

Model selection
• Overfitting: The model has too many parameters relative to the
number of observations
⋆ large estimation uncertainty
⋆ poor predictions
• Underfitting: The model has too few parameters relative to the
number of observations
⋆ poor fit to data
⋆ does not use all information in the data
⋆ poor predictions
• Methods for model selection
⋆ Cross-validation - mentioned in space-time modeling course
⋆ AIC
⋆ BIC
26

Model selection by AIC and BIC
• Want as best fit as possible
with as few parameters as possible
• p=number of parameters and n=number of observations
• AIC = - 2 log likelihood + 2 p
• BIC = - 2 log likelihood + log(n) p
27

Ex: Model selection using BIC
> glmobj1 glmobj2 glmobj12 glmobj123
> bic bic[1] bic[2] bic[3] bic[4]
> bic
[1] 1113.910 1339.247 1104.025 1106.019
• The model with x 1 and x 2 has the lowest BIC value
and is selected
28