20.04.2013 Views

Documentation of the Evaluation of CALPUFF and Other Long ...

Documentation of the Evaluation of CALPUFF and Other Long ...

Documentation of the Evaluation of CALPUFF and Other Long ...

SHOW MORE
SHOW LESS

Create successful ePaper yourself

Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.

AM<br />

∩ AP<br />

FMS = × 100%<br />

(2‐2)<br />

A ∪ A<br />

M<br />

P<br />

The more <strong>the</strong> predicted <strong>and</strong> measured tracer clouds overlap one ano<strong>the</strong>r, <strong>the</strong> greater <strong>the</strong> FMS<br />

values are. A high FMS value corresponds to better model performance, with a perfect model<br />

achieving a 100% FMS score.<br />

Additional spatial performance measures <strong>of</strong> Probability Of Detection (POD), False Alarm Rate<br />

(FAR), <strong>and</strong> Threat Score (TS) are also used. Typically used as a method for meteorological<br />

forecast verification, <strong>the</strong>se three interrelated statistics are useful descriptions <strong>of</strong> an air quality<br />

model’s ability to spatially forecast a certain condition. The forecast condition for <strong>the</strong> model is<br />

<strong>the</strong> predicted concentration above a user‐specified threshold (at <strong>the</strong> 0.1 ngm ‐3 (100 pgm ‐3 ) level<br />

for ATMES‐II study). In <strong>the</strong>se equations:<br />

• “a” represents <strong>the</strong> number <strong>of</strong> times a condition that has been forecast, but was not<br />

observed (false alarm)<br />

• “b” represents <strong>the</strong> number <strong>of</strong> times <strong>the</strong> condition was correctly forecasted (hits)<br />

• “c” represents <strong>the</strong> number <strong>of</strong> times <strong>the</strong> nonoccurrence <strong>of</strong> <strong>the</strong> condition is correctly<br />

forecasted (correct negative); <strong>and</strong><br />

• “d” represents <strong>the</strong> number <strong>of</strong> times that <strong>the</strong> condition was observed but not forecasted<br />

(miss).<br />

The FAR (Equation 2‐3) is described as a measure <strong>of</strong> <strong>the</strong> percentage <strong>of</strong> times that a condition<br />

was forecast, but was not observed. The range <strong>of</strong> <strong>the</strong> score is 0 to 1 or 0% to 100%, with <strong>the</strong><br />

ideal FAR score <strong>of</strong> 0 or 0% (i.e., <strong>the</strong>re are observed tracer concentrations at a monitor/time<br />

every time <strong>the</strong> model predicts <strong>the</strong>re is a tracer concentration at that monitor/time).<br />

⎛ a ⎞<br />

FAR = ⎜ ⎟×<br />

100%<br />

(2‐3)<br />

⎝ a + b ⎠<br />

The POD is a statistical measure which describes <strong>the</strong> fraction <strong>of</strong> observed events <strong>of</strong> <strong>the</strong><br />

condition forecasted was correctly forecasted. Equation 2‐4 shows that POD is defined as <strong>the</strong><br />

ratio <strong>of</strong> “hits” to <strong>the</strong> sum <strong>of</strong> “hits” <strong>and</strong> “misses.” The range <strong>of</strong> <strong>the</strong> POD score is 0 to 1 (or 0%to<br />

100%), with <strong>the</strong> ideal score <strong>of</strong> 1 (or 100%).<br />

⎛ b ⎞<br />

POD = ⎜ ⎟×<br />

100%<br />

(2‐4)<br />

⎝ b + d ⎠<br />

The TS (Equation 2‐5) is described as <strong>the</strong> measure describing how well correct forecasts<br />

corresponded to observed conditions. The TS does not consider correctly forecasted negative<br />

conditions, but penalizes <strong>the</strong> score for both false alarms <strong>and</strong> misses. The range <strong>of</strong> <strong>the</strong> TS is <strong>the</strong><br />

same as <strong>the</strong> POD, ranging from 0 to 1 (0% to 100%), with <strong>the</strong> ideal score <strong>of</strong> 1 (100%).<br />

16

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!