D5 Annex report WP 3: ETIS Database methodology ... - ETIS plus
D5 Annex report WP 3: ETIS Database methodology ... - ETIS plus D5 Annex report WP 3: ETIS Database methodology ... - ETIS plus
D5 Annex WP 3: DATABASE METHODOLOGY AND DATABASE USER MANUAL – FREIGHT TRANSPORT DEMAND cases where also no national sources can be found UN trade data will be used having the disadvantage that no mode information is included. In the trade data every trade relation is being registered from the export as well as the import side. These trade data sources, whose main function is to provide information relating to the value of trade, need to be modified before the information can be related to the volume of trade, even where volumes are recorded by the relevant customs authority. Corrections have to be applied due to incompatible measurement units (litres, kilograms, square metres), or due to data errors, often of significant magnitude. The first problem (incompatible units) can be solved by applying conversion ratios for the same commodity groups in instances where the ratio is known. The second problem (data outliers) can be solved by applying a smoothing technique to a time series of data. Such a technique has been applied with the MDS Transmodal trade forecasting model. The technique works by scanning the COMEXT data for a single trade flow (e.g. French exports of SITC 56 in tonnes to Italy) for quarterly time periods (of several years). The smoothing software samples four data points for each year and calculates the mean and the standard deviation for that year. It then compares each year's mean and standard deviation with all the others. Then if there are any years with unusual levels of variance, they are investigated by the software, and according to certain thresholds individual quarterly values may be marked as outliers and the software will replace them with interpolated values. It means that very erratic series will be left untouched, but erratic sequences within normally stable series will be corrected. Furthermore there will be adjustment required due to the resale of (bulk) commodities. For instance there is a significant amount of ores traded from the Netherlands to Germany according to COMEXT. These ores however do not originate from the Netherlands but are resold after being imported in the Netherlands, which implies that in fact this flow of ores between the Netherlands and Germany is just one link from the total transport chain. In the NEAC construction history many of these cases have been identified and can be traced back by comparing the NEAC forecast up to the year 2000 with the COMEXT 2000 data. The large outliers are being manually analysed. This procedure will be applied on top of the earlier mentioned procedure. For a future update of the ETIS reference database freight OD the OD matrix currently being developed can then be used for identification of outliers. 22 Document2 27 May 2004
D5 Annex WP 3: DATABASE METHODOLOGY AND DATABASE USER MANUAL – FREIGHT TRANSPORT DEMAND The transport mode is registered at the border of the country in most trade sources and at the border of the EU for the extra EU trade in COMEXT; as a result it is possible to estimate the part that is transhipped onto another mode. When the trade statistics show that a flow leaves Spain by sea and enters Poland by road, it can be concluded that somewhere transhipment has taken place. In this phase all direct transport without transhipment and indirect transport with transhipment is registered. A difference in definition appears here since for the extra EU trade the mode is not anymore registered at the border of the countries but at the border of the EU. Specific solutions will be analysed amongst which the option of estimation of the country border mode by assignment in following steps. 4.5.2 Phase II Including transhipment regions on the basis of transhipment statistics The identification of transhipment regions is taking place with the help of the available statistics originating from the national transhipment sources or ports and terminal. The inclusion of inland terminal information will be considered here as an experiment since no proof of concept is available. In this step two transhipment points will be included for intra ETIS reference database core area short sea flows. All collected port flows will be combined into one database. Here double countings have to be eliminated in the cases where for two ports transhipment data are available and where these ports have a connecting service; these flows are then registered at both ports. The port flows are then included in the trade database of step one taking account of all information included in the data (origin, destination, commodity, modes) and again removing all the double countings. This way the trade volumes on country to country level are the same as in step 1, but it is known whether transhipment takes place along the route, where this takes place and from what mode to what other mode. 4.5.3 Phase III Regional division of countrytocountry totals The first two steps resulted into information ranging from both the country of origin to the country of destination. The regional detail is added to the database by means of the different sources dividing the trade flows over the regions in a country. Countries for which this can be done we will call A countries. For the other countries to be called B countries, regionalisation can be performed by using domestic transport statistics (for instance New Cronos). These domestic transport statistics often only show us the total flows arriving or departing from a region. The remaining countries will be called C countries. For this last category estimation procedures will be applied which are developed in ODESTIM and which make use of socioeconomic data (see section 6.6). Document2 27 May 2004 23
- Page 1: D5 Annex report WP 3: ETIS Databa
- Page 4 and 5: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 7: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 10 and 11: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 12 and 13: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 14 and 15: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 16 and 17: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 19 and 20: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 21: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 25 and 26: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 27 and 28: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 29: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 32 and 33: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 34 and 35: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 36 and 37: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 38 and 39: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 40 and 41: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 43 and 44: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 45 and 46: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 47 and 48: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 49 and 50: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 51 and 52: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 53 and 54: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 55 and 56: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 57 and 58: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 59 and 60: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 61 and 62: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 63 and 64: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 65 and 66: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 67: D5 Annex WP 3: DATABASE METHODOLOGY
- Page 71 and 72: D5 Annex WP 3: DATABASE METHODOLOGY
<strong>D5</strong> <strong>Annex</strong> <strong>WP</strong> 3: DATABASE METHODOLOGY AND DATABASE USER MANUAL –<br />
FREIGHT TRANSPORT DEMAND<br />
cases where also no national sources can be found UN trade data will be used having the<br />
disadvantage that no mode information is included. In the trade data every trade relation is being<br />
registered from the export as well as the import side.<br />
These trade data sources, whose main function is to provide information relating to the value of<br />
trade, need to be modified before the information can be related to the volume of trade, even<br />
where volumes are recorded by the relevant customs authority. Corrections have to be applied<br />
due to incompatible measurement units (litres, kilograms, square metres), or due to data errors,<br />
often of significant magnitude. The first problem (incompatible units) can be solved by applying<br />
conversion ratios for the same commodity groups in instances where the ratio is known. The<br />
second problem (data outliers) can be solved by applying a smoothing technique to a time series<br />
of data. Such a technique has been applied with the MDS Transmodal trade forecasting model.<br />
The technique works by scanning the COMEXT data for a single trade flow (e.g. French exports<br />
of SITC 56 in tonnes to Italy) for quarterly time periods (of several years). The smoothing<br />
software samples four data points for each year and calculates the mean and the standard<br />
deviation for that year. It then compares each year's mean and standard deviation with all the<br />
others. Then if there are any years with unusual levels of variance, they are investigated by the<br />
software, and according to certain thresholds individual quarterly values may be marked as<br />
outliers and the software will replace them with interpolated values. It means that very erratic<br />
series will be left untouched, but erratic sequences within normally stable series will be<br />
corrected.<br />
Furthermore there will be adjustment required due to the resale of (bulk) commodities. For<br />
instance there is a significant amount of ores traded from the Netherlands to Germany according<br />
to COMEXT. These ores however do not originate from the Netherlands but are resold after<br />
being imported in the Netherlands, which implies that in fact this flow of ores between the<br />
Netherlands and Germany is just one link from the total transport chain. In the NEAC<br />
construction history many of these cases have been identified and can be traced back by<br />
comparing the NEAC forecast up to the year 2000 with the COMEXT 2000 data. The large<br />
outliers are being manually analysed. This procedure will be applied on top of the earlier<br />
mentioned procedure. For a future update of the <strong>ETIS</strong> reference database freight OD the OD<br />
matrix currently being developed can then be used for identification of outliers.<br />
22<br />
Document2<br />
27 May 2004
Change language