Documentation of the Evaluation of CALPUFF and Other Long ...

Factor of α (FAα): FAα represents the percentage of predicted values that are within a factor of
α, where we have used α = 2 or 5. As with FOEX, in FAα all 0‐0 pairs are excluded.
Normalized Mean Squared Error (NMSE): Normalized mean squared error is the average of the
square of the differences divided by the product of the means. NMSE gives information about
the deviations, but does not yield estimations of model over‐prediction or under‐prediction.
( ) 2
1 NMSE = ∑ Pi
− M i
(2‐9)
N PM
Pearson’s Correlation Coefficient (PCC): Also referred to as the linear correlation coefficient, its
value ranges between ‐1.0 and +1.0. A value of +1.0 indicates “perfect positive correlation” or
having all pairings of (Mi, Pi) lay on straight line on a scatter diagram with a positive slope.
Conversely, a value of ‐1.0 indicates “perfect negative correlation” or having all pairings of (Mi,
Pi) lie on a straight line with a negative slope. A value of near 0.0 indicates the clear absence of
relationship between the model predictions and observed values.
R =
⎡
⎢
⎣
∑
( y − y0
= [ x x0]
α)
⎡N − ⎤
FAα
=
⎢
⎥
× 100
(2‐8)
⎣ N ⎦
∑
i
( M − M ) • ( P − P)
i
2 ⎤⎡
2 ⎤
( M i − M ) ⎥⎢
∑(
Pi
− P)
⎥⎦
⎦⎣
Fractional Bias (FB): Calculated as the mean difference in prediction minus observation pairings
divided by the average of the predicted and observed values.
( P M )
FB = 2 B +
(2‐11)
Kolmogorov‐Smirnov Parameter (KS): The KS parameter is defined as the maximum difference
between two cumulative distributions. The KS parameter provides a quantitative estimate
where C is the cumulative distribution of the measured and predicted concentrations over the
range of k. The KS is a measure of how well the model reproduces the measured concentration
distribution regardless of when or where it occurred. The maximum difference between any
two distributions cannot be more than 100%.
( M ) C(
P )
KS = MaxC
−
(2‐12)
k
RANK: Given the large number of metrics, a single measure describing the overall performance
of a model could be useful. Stohl et al. (1998) evaluated many of the above measures and
18
i
k
(2‐10)