Documentation of the Evaluation of CALPUFF and Other Long ...

14. Across all the spatial statistics, EXP6A performs the best with EXP4A, EXP4B, EXP6B and
4KM_MMIF next best and 12KM_MMIF being worst.
80%
70%
60%
50%
40%
30%
20%
10%
0%
35%
30%
25%
20%
15%
10%
5%
0%
EXP 4A
EXP 4A
Figure of Metric in Space (FMS)
(Perfect = 100%)
EXP 4B
EXP 4B
EXP 4C
EXP 4D
EXP 6A
EXP 6B
EXP 6C
EXP 6D
Probability of Detection (POD)
(Perfect = 100%)
EXP 4C
EXP 4D
EXP 6A
EXP 6B
EXP 6C
EXP 6D
4K MMIF
12K MMIF
4K MMIF
12K MMIF
50%
40%
30%
20%
10%
0%
Figure 5‐14. Spatial model performance statistics for the CTEX5 CALPUF sensitivity tests that
used the 12 and 4 km MM5 data.
The lowest error of the 12 km MM5 CALPUFF sensitivity tests is given by EXP4B and EXP6A‐C,
with 12KM_MMIF having the highest error (Figure 5‐15). EXP6B has the lowest bias follows by
EXP6C and EXP4B, with 12KM_MMIF having the largest bias. EXP6A and EXP6B have the most
model predictions within a factor of 2 of the observations and EXP4C has the most within a
factor of 5. The CALMET/CALPUFF correlation coefficients range from 0.57 to 0.76 with EXP4A
(0.76) and EXP6C (0.75) have the highest values and EXP6B (0.57) having the lowest value. The
4KM_MMIF has an even lower correlation coefficient (0.48) with the 12KM_MMIF having no to
slight anti‐correlation with the observed values (‐0.07). According to the RANK composite
statistics the best performing 12 km CALPUFF sensitivity test in EXP6C (2.19) followed by EXP6A
(2.02) and EXP4A (1.98) with 12KM_MMIF (1.28) performing worst.
92
100%
80%
60%
40%
20%
0%
EXP 4A
EXP 4A
EXP 4B
EXP 4B
False Alarm Rate (FAR)
(Perfect = 0%)
EXP 4C
EXP 4C
EXP 4D
EXP 6A
EXP 6B
EXP 6C
Threat Score (TS)
(Perfect = 100%)
EXP 4D
EXP 6A
EXP 6B
EXP 6C
EXP 6D
4K MMIF
EXP 6D
12K MMIF
4K MMIF
12K MMIF