Documentation of the Evaluation of CALPUFF and Other Long ...

that performance evaluation be based on comparisons of the full set of predicted/observed
data pairs as well as the highest predicted and observed values per event and the highest N
values (e.g., N=10) unpaired in space or time that represents the highest end of the
concentration distribution.
Six of the eight LRT models were applied to both the GP80 and SRL75 experiments. The ARRPA
model could only be applied to the GP80 database and the MTDDIS model could only be
applied to the SRL75 database. Model performance was generally consistent between the two
tracer databases and was characterized by three features:
• A spatial offset of the predicted and observed patterns.
• A time difference between the predicted and observed arrival of the plumes to the
receptors.
• A definite angular offset of the predicted and observed plumes that could be as much as
20‐45 degrees.
The LRT models tended to underestimate the horizontal spreading of the plume at ground level
resulting in too high peak (centerline) concentrations when compared to the observations. For
the Lagrangian models this is believed to be due to using sigma‐y dispersion (Turner) curves
that are representative of near‐source and are applied for longer (> 50 km) downwind
distances. The spatial and angular offsets resulted in poor correlations and large bias and error
between the predicted and observed tracer concentrations when paired by time and location.
However, when comparing the maximum predicted and observed concentrations unmatched
by time and location, the models performed much better. For example, the average of the
highest 25 predicted and observed concentrations (unpaired in location and time) were within
a factor of two for six of the eight models evaluated (MESOPUFF, MESOPLUME, MESOPLUME,
MTDDIS, ARRPA and RTM‐II). The study concluded that the LRT models’ observed tendency to
over‐predict the observed peak concentrations errs on the conservative side for regulatory
applications. However, this over‐prediction must be weighed against the general tendency of
those models to underestimate horizontal spreading and to predict a plume pattern that is
spatially offset from the observed data.
2.3.2 Rocky Mountain Acid Deposition Model Assessment Project – Western Atmospheric
Deposition Task Force
A second round of LRT model evaluations was conducted as part of the Rocky Mountain Acid
Deposition Model Assessment (EPA, 1990). In this study, the eight models from the 1986
evaluation were compared against a newer model, the Acid Rain Mountain Mesoscale Model
(ARM3) (EPA, 1988). The statistical evaluation considered data paired in time/space and also
unpaired in time/space equally. In this study, it was found that the MESOPUFF‐II (Scire et al.,
1984a, and 1984b) model performed best when using unpaired data, and that the ARM3 model
performed best when using paired data. A final model score was assigned on the basis of a
model’s performance relative to the others in each of the areas (paired in time/space, unpaired
in time/space, and paired in time, not space) for each of two tracer releases considered.
The primary objective was to assemble a mesoscale air quality model based primarily on
models or model components available at the time for use by state and federal agencies to
assess acid deposition in the complex terrain of the Rocky Mountains.
8