D5 Annex report WP 3: ETIS Database methodology ... - ETIS plus

D5 Annex WP 3: DATABASE METHODOLOGY AND DATABASE USER MANUAL –
FREIGHT TRANSPORT DEMAND
The COMEXT database for the Year 2000 has been used for the testing phase of ETIS. The
first test has considered whether it is necessary to apply additional error checking routines to
avoid the inclusion of data errors in the subsequent O/D matrices. This has been achieved by
comparing ‘smoothed’ data to the raw data to measure the impact of erratic values in the
database.
A technique for identifying outliers (errors or “erratics”) has been developed within the MDS
Transmodal trade forecasting model, taking into account a long time series of trade data.
During this process, the COMEXT data is converted into time series vectors for individual trade
flows (e.g. French exports of SITC 56 in tonnes to Italy) for quarterly time periods covering
approximately fifteen years. The smoothing software samples four data points for each year and
calculates the mean and the standard deviation for that year. It then compares each year's mean
and standard deviation with all the others. Then if there are any years with unusual levels of
variance, they are investigated by the software, and according to certain thresholds individual
quarterly values may be marked as outliers and the software will replace them with interpolated
values. It means that normally erratic series will be left untouched, but erratic points or
sequences within normally stable series will be changed. Every year, new data is collected, and
the process is repeated, so it is possible that what is regarded as an outlier may change over time
as the software learns more about the time series.
The use of the smoothing algorithm can be illustrated, by comparing the smoothed data to the
original data.
In 2000, imports into EU countries amounted to 2.501 billion tonnes according to COMEXT.
After smoothing the estimate was 2.391 billion tonnes, a change of only 4%. The largest
absolute error in a single 2 digit SITC category is 13 million tonnes, for SITC 33, petroleum.
However this is only a 2% difference within that category. The largest percentage error is for
SITC 83, travel goods, with a 63% percent difference. However this only amounts to an
absolute difference of 1.133 million tonnes. Most of the difference can be traced to a figure of
1.079 million tonnes for travel goods between the UK and Germany.
Looking at the same trade flow using German export data a total of 0.001 million tonnes can be
found, a level that agrees more readily with the smoothed data. This difference (1000 times) is
untypical however. Absolute differences are typically about 1 million tonnes per 2 digit SITC
category, and relative differences are typically about 6%. It should also be noted that
differences between smoothed and un­smoothed series do not necessarily imply that the unsmoothed
series contains errors, only values that are unlikely to be repeated.
At these levels, and given the scope of ETIS, the potential impacts of measurement errors are
not alarming, particularly when the annual version of COMEXT is used. However, the example
related above does suggest that a high level comparison between the trade data used within
ETIS and the data based upon a smoothed quarterly time series will reveal a small number of
44
Document2
27 May 2004