Documentation of the Evaluation of CALPUFF and Other Long ...
Documentation of the Evaluation of CALPUFF and Other Long ... Documentation of the Evaluation of CALPUFF and Other Long ...
6.4.1.1 Spatial Analysis of Model Performance Four spatial analysis model performance statics have been identified and are discussed in this section: FMS, FAR, POD and TS. Figure 6‐5 displays the FMS spatial analysis performance metrics for the five LRT models and the ETEX tracer study field experiment. Recall that the FMS statistic is define as the overlap divided by the union of the predicted and observed tracer clouds with a perfect model receiving an FMS score of 100%. 60% 50% 40% 30% 20% 10% 0% Figure of Metric in Space (FMS) (Perfect = 100%) CALPUFF SCIPUFF HYSPLIT FLEXPART CAMx Figure 6‐5. Figure of Merit in Space (FMS) statistical performance metric for the five LRT models and the ETEX tracer field experiment. Figure 6‐6 displays the False Alarm Rate (FAR) performance metrics. The FAR metric is defined by the number of times that a tracer concentration was predicted to occur at a monitor‐time when no tracer was observed (i.e., a miss) divided by the number of times a tracer was predicted to occur at a monitor‐time (i.e., sum of misses and hits); a perfect model (i.e., one that had no misses) would have a FAR score of 0%. 111
70% 60% 50% 40% 30% 20% 10% 0% Figure 6‐6. False Alarm Rate (FAR) statistical performance metric for the five LRT models and the ETEX tracer field experiment. The Probability of Detection (POD) performance statistic is defined as the percent of the time the predicted and observed tracer both occurred at a monitor‐time (i.e., a hit of tracer concentrations greater than 1 ngm ‐3 ) divided by the number of times that the tracer was observed at any monitor‐time (i.e., sum of hits and misses); a perfect model POD score would be 100% (i.e., anytime there was observed tracer at a monitor there was also predicted tracer at the monitor). 70% 60% 50% 40% 30% 20% 10% 0% False Alarm Rate (FAR) (Perfect = 0%) CALPUFF SCIPUFF HYSPLIT FLEXPART CAMx Probability of Detection (POD) (Perfect = 100%) CALPUFF SCIPUFF HYSPLIT FLEXPART CAMx Figure 6‐7. Probability of Detection (POD) statistical performance metric for the five LRT models and the ETEX tracer field experiment. The Threat Score (TS) is the ratio of the number of times that a tracer is both predicted and observed at a monitor‐time at the same time (i.e., common hits among the predictions and 112
- Page 97 and 98: Table 4‐4. CALPUFF parameters use
- Page 99 and 100: Table 4‐8. CALPUFF/MMIF sensitivi
- Page 101 and 102: the fitted Gaussian plume is not a
- Page 103 and 104: Figure 4‐2. Comparison of predict
- Page 105 and 106: Figure 5‐1. Location of Dayton an
- Page 107 and 108: MM5 runs, the first without FDDA (i
- Page 109 and 110: Table 5‐3. MM5 sensitivity tests
- Page 111 and 112: Table 5‐6. Definition of the CALM
- Page 113 and 114: performance at the monitor location
- Page 115 and 116: 35% 30% 25% 20% 15% 10% 5% 0% 35% 3
- Page 117 and 118: 40% 35% 30% 25% 20% 15% 10% 5% 0% F
- Page 119 and 120: 40% 35% 30% 25% 20% 15% 10% 5% 0% E
- Page 121 and 122: 5.4.1.4 Comparison of CALPUFF CTEX3
- Page 123 and 124: 0.48 0.36 0.24 0.12 0 ‐0.12 16% 1
- Page 125 and 126: CTEX3 discussed in Section 5.4.1. A
- Page 127 and 128: CALPUFF sensitivity simulations are
- Page 129 and 130: 14. Across all the spatial statisti
- Page 131 and 132: sensitivity tests. The “B” seri
- Page 133 and 134: ‐0.1 ‐0.2 0 0.8 0.7 0.6 0.5 0.4
- Page 135 and 136: 6.0 1994 EUROPEAN TRACER EXPERIMENT
- Page 137 and 138: Figure 6‐2a. Surface synoptic met
- Page 139 and 140: Figure 6‐3a. Distribution of the
- Page 141 and 142: 36 kilometers and the vertical stru
- Page 143 and 144: splitting flag near sunset (hour 17
- Page 145 and 146: experienced during the original ETE
- Page 147: 2 1 0 ‐1 ‐2 3 2 1 0 23‐Oct 23
- Page 151 and 152: Figure 6‐9. Factor of Exceedance
- Page 153 and 154: eceiving a 0.0 score. Figure 6‐13
- Page 155 and 156: Table 6‐1. Summary of model ranki
- Page 157 and 158: plume spread and observed surface c
- Page 159 and 160: Figure 6‐16c. Comparison of spati
- Page 161 and 162: • NoPiG: The tracer emissions wer
- Page 163 and 164: Using the NMSE statistical performa
- Page 165 and 166: 6.4.3.2 Effect of PiG on Model Perf
- Page 167 and 168: 80 70 60 50 40 30 20 10 0 1 2 3 4 5
- Page 169 and 170: Table 6‐3. Summary of CALPUFF puf
- Page 171 and 172: 0.2 0.18 0.16 0.14 0.12 0.1 0.08 0.
- Page 173 and 174: Figure 6‐ ‐22 displays the t sp
- Page 175 and 176: Figure 6‐ ‐23a. Global model pe
- Page 177 and 178: Figure 6‐24. Figure of Merit (FMS
- Page 179 and 180: 7.0 REFERENCES Anderson, B. 2008. T
- Page 181 and 182: EPA, 1984: Interim Procedures for E
- Page 183 and 184: Mlawer, E.J., S.J. Taubman, P.D. Br
- Page 185 and 186: 148 Appendix A Evaluation of the MM
- Page 187 and 188: Table A‐1. Wind speed and wind di
- Page 189 and 190: Table A‐3. Definition of the CTEX
- Page 191 and 192: Figure A‐ ‐1. Wind speed bias (
- Page 193 and 194: Figure A‐ ‐3. Humidity bias and
- Page 195 and 196: Table A‐5. Comparison of CTEX5 MM
- Page 197 and 198: B.1 CALMET MODEL EVALUATION TO IDEN
70%<br />
60%<br />
50%<br />
40%<br />
30%<br />
20%<br />
10%<br />
0%<br />
Figure 6‐6. False Alarm Rate (FAR) statistical performance metric for <strong>the</strong> five LRT models <strong>and</strong><br />
<strong>the</strong> ETEX tracer field experiment.<br />
The Probability <strong>of</strong> Detection (POD) performance statistic is defined as <strong>the</strong> percent <strong>of</strong> <strong>the</strong> time<br />
<strong>the</strong> predicted <strong>and</strong> observed tracer both occurred at a monitor‐time (i.e., a hit <strong>of</strong> tracer<br />
concentrations greater than 1 ngm ‐3 ) divided by <strong>the</strong> number <strong>of</strong> times that <strong>the</strong> tracer was<br />
observed at any monitor‐time (i.e., sum <strong>of</strong> hits <strong>and</strong> misses); a perfect model POD score would<br />
be 100% (i.e., anytime <strong>the</strong>re was observed tracer at a monitor <strong>the</strong>re was also predicted tracer<br />
at <strong>the</strong> monitor).<br />
70%<br />
60%<br />
50%<br />
40%<br />
30%<br />
20%<br />
10%<br />
0%<br />
False Alarm Rate (FAR)<br />
(Perfect = 0%)<br />
<strong>CALPUFF</strong> SCIPUFF HYSPLIT FLEXPART CAMx<br />
Probability <strong>of</strong> Detection (POD)<br />
(Perfect = 100%)<br />
<strong>CALPUFF</strong> SCIPUFF HYSPLIT FLEXPART CAMx<br />
Figure 6‐7. Probability <strong>of</strong> Detection (POD) statistical performance metric for <strong>the</strong> five LRT<br />
models <strong>and</strong> <strong>the</strong> ETEX tracer field experiment.<br />
The Threat Score (TS) is <strong>the</strong> ratio <strong>of</strong> <strong>the</strong> number <strong>of</strong> times that a tracer is both predicted <strong>and</strong><br />
observed at a monitor‐time at <strong>the</strong> same time (i.e., common hits among <strong>the</strong> predictions <strong>and</strong><br />
112