Documentation of the Evaluation of CALPUFF and Other Long ...

6.4.3.2 Effect of PiG on Model Performance
Whether better model performance is obtained using the PiG module or not frequently
depends on the statistical metric being analyzed and the CAMx model configuration (vertical
diffusion algorithm and horizontal advection solver). However, whether the PiG is used or not
has very little difference on the rankings of the CAMx model performance using the alternative
vertical mixing and horizontal advection approaches. In general, it appears that the CAMx
model performance without the PiG is performing slightly better than its performance using the
PiG.
The spatial performance statistics are sometimes improved and sometimes degraded when the
PiG module is invoked. For the global statistics, the PCC performance statistic is degraded by ‐
11% to ‐37% (‐0.03 to ‐0.13 points) when the PiG module is invoked. Similarly, use of the PiG
versus NoPiG module increases (degrades) the FB metric by 5 to 18 percent and also increases
(degrades) the NMSE metrics for all model configurations.
Table 6‐2 summarized the RANK model performance statistic for the different CAMx model
configurations with and without the PiG module. For each model vertical diffusion/horizontal
advection configuration, using the PiG module always results in slightly lower RANK statistics
that are from ‐3.9% to ‐8.5% lower than when the PiG module is not used. The ranking of the
top four CAMx vertical diffusion/horizontal advection configurations remains unchanged
whether the PiG module is used or not. And by far the most important parameter examined in
regards to the RANK model performance statistics for the ETEX experiment in the CAMx
sensitivity tests is the vertical mixing algorithm, with the CMAQ Kz parameterization producing
the best four RANK model performance statistics out of the 16 sensitivity tests: (1)
NoPiG/CMAQ/PPM; (2) NoPiG/CMAQ/Bott; (3) PiG/CMAQ/PPM; and (4) PiG/CMAQ/Bott.
Table 6‐2. CAMx RANK model performance statistic and model rankings for different model
configurations with and without using the PiG subgrid‐scale puff model.
Without PiG Module With PiG Module PiG‐NoPiG
Model
Model
Model
Configuration RANK Ranking RANK Ranking ΔRANK Percent
OB70/BOTT 1.60 7 a 1.53 6 a
‐0.07 ‐4.4%
OB70/PPM 1.66 4 1.55 4 ‐0.11 ‐6.6%
TKE/BOTT 1.65 5 1.51 7 ‐0.14 ‐8.5%
TKE/PPM 1.70 3 1.56 3 ‐0.14 ‐8.2%
ACM2/BOTT 1.54 8 1.48 8 ‐0.06 ‐3.9%
ACM2/PPM 1.60 6 a 1.53 5 a
‐0.07 ‐4.4%
CMAQ/BOTT 1.90 2 1.76 2 ‐0.14 ‐7.4%
CMAQ/PPM
a
tied
1.94 1 1.80 1 ‐0.14 ‐7.2%
128