Traitement et analyse de séries chronologiques continues de ...

Traitement et analyse de séries chronologiques continues de ... Traitement et analyse de séries chronologiques continues de ...

theses.insa.lyon.fr
from theses.insa.lyon.fr More from this publisher
14.09.2014 Views

Annexes Secondly, the determination of correlation functions between continuously measured turbidity and sampled TSS and COD concentrations. These correlation functions are both site and sensor specific. To ensure their reliability, outliers in calibration data sets must be detected and specific regression methods which account for uncertainties in all variables need to be applied. A set of tools have been developed in this research for (i) raw data correction and evaluation of uncertainty, (ii) data validation and (iii) determination of correlation functions. However, converting the mess of very long continuous series of raw data to estimates of the event pollutant mass with their uncertainties remains a time consuming task if software assistance is not available. This paper describes the most important steps of an integrated and automated data processing methodology, particularly focused on new or adapted methods. The methodology has been tested and validated, using a five-year period data collected from a separate stormwater sewer system in Lyon, France. The main original characteristics of the proposed methodology described in the following sections are as follows: (i) implementation of a variance analysis method to automatically select the most pertinent calibration function for each sensor, (ii) application of an extended version of the Williamson regression to explicitly account for uncertainties in both variables, (iii) systematic and automated evaluation of the uncertainties from recorded raw data to final variables of interest like pollutant loads, (iv) explicit accounting for in situ uncertainties in addition to sensor uncertainties. METHODOLOGY The proposed methodology, when applied for the first time to a data set of time series, should be implemented in the following order: 1. calibration of sensors and determination of calibration functions 2. data correction and estimation of uncertainties in corrected data 3. automated data pre-validation by application of a set of parametric tests 4. final data validation by an operator 5. calculation of discharge and concentrations of TSS and COD, and of their uncertainties 6. calculation of event TSS and COD loads and of their uncertainties. Next applications start at step number 2, until calibration functions are updated according to periodic verifications and re-calibration of sensors. A previously processed data set can be re-processed from any step if revised settings are implemented for this step. In practice, the software package which has been developed to implement the methodology includes log books and storage of raw and intermediate data sets. The following sub-sections briefly describe the key principles applied in these main steps. Sensor calibration This step comprises the calibration of all sensors used and the determination of the information needed for further data correction and the calculation of their standard uncertainties. Calibration functions are calculated considering two main sources of uncertainties, namely (i) sensor uncertainties evaluated during the calibration procedure itself, and (ii) field measurement uncertainties due to in situ measurement conditions which are evaluated by means of local observations and expertise.

Annexes Sensor uncertainties. Calibration function is chosen either a straight line, second or third order polynomial function for different sensor types based on Fischer-Snedecor statistical test. With regards to the variance of measurements, the ordinary least squares regression for constant variance or the Williamson regression accounting for uncertainties in both standards and sensor response for non constant variance are used. (Bertrand-Krajewski, 2004; Bertrand-Krajewski et al., 2007). In both regression types, uncertainties in function parameters can be calculated either analytically or by Monte Carlo simulations. The sensor uncertainty is estimated from the variance of the calibration measurements: the ordinary least squares regression is set a constant sensor uncertainty as the maximum value of the observed variance, and the Williamson regression is assigned a variable uncertainty according to the observed variability of the variance along the measurement range. Field measurement uncertainties. The in situ uncertainty is both sensor and site specific. Its value should be evaluated from local measurements and expertise. For instance, in the example detailed hereafter, turbidity is not directly measured in the sewer but in a shelter where a transit flume is continuously supplied with wastewater from the sewer by means of a peristaltic pump. In this case, the representativeness of the wastewater in the flume and the varying position of the mobile pump intake in the sewer should be considered as additional sources of uncertainty. These were estimated to be approximately equal to 10 % of the measured value. Another example is related to the water level measurements. The sensor uncertainty of an ultrasonic probe is evaluated to be equal to 3 mm according to calibration experiments. As (i) the sensor is not perfectly installed and located in the sewer, and (ii) the free surface is not flat but experiences waves of height ranging from 1.5 cm in dry weather up to approximately 3 cm in wet weather events, the in situ standard uncertainty in the water level is set equal to 7.5 mm - 15 mm depending on the water level. Data correction This step comprises the correction of raw data at each time step according to the sensor calibration functions and the calculation of their standard uncertainties from the information provided in the calibration step. Corrected values Xˆ i of raw measurements X i at time step i are estimated by the inverse calibration function, either by direct analytical calculation (in case of calibration functions of degrees 1 and 2) or by numeric minimization (in less frequent case of calibration function of degree 3), with addition of an offset value when necessary. The total standard uncertainty in the corrected values is calculated by: u ˆ 2 ˆ tot Xi um Xi 2 usiteX ˆ i 2 where mX i uncertainties calculated according to the law of propagation of uncertainties (LPU), and (1) u ˆ is the measurement standard uncertainty accounting for sensor and calibration usite Xˆ i is the in situ standard uncertainty. The standard uncertainty and the LPU approach are applied according to international standards for the expression of uncertainty in measurements (ENV 1999, ISO, 2009). Selection of pre-validation tests and setting of test parameters This step aims to select the tests to be applied (see Figure 1) and to set the parameters required for the pre-validation tests. The pre-validation method is based on the method initially developed by Mourad and Bertrand-Krajewski (2002). Three pre-validation marks are used: 1 (valid data), 2 (doubtful data) or 3 (incorrect data). Eight pre-validation tests are available and selected by the user according to the

Annexes<br />

Sensor uncertainties. Calibration function is chosen either a straight line, second or third or<strong>de</strong>r<br />

polynomial function for different sensor types based on Fischer-Sne<strong>de</strong>cor statistical test. With regards<br />

to the variance of measurements, the ordinary least squares regression for constant variance or the<br />

Williamson regression accounting for uncertainties in both standards and sensor response for non<br />

constant variance are used. (Bertrand-Krajewski, 2004; Bertrand-Krajewski <strong>et</strong> al., 2007). In both<br />

regression types, uncertainties in function param<strong>et</strong>ers can be calculated either analytically or by Monte<br />

Carlo simulations. The sensor uncertainty is estimated from the variance of the calibration<br />

measurements: the ordinary least squares regression is s<strong>et</strong> a constant sensor uncertainty as the<br />

maximum value of the observed variance, and the Williamson regression is assigned a variable<br />

uncertainty according to the observed variability of the variance along the measurement range.<br />

Field measurement uncertainties. The in situ uncertainty is both sensor and site specific. Its value<br />

should be evaluated from local measurements and expertise. For instance, in the example d<strong>et</strong>ailed<br />

hereafter, turbidity is not directly measured in the sewer but in a shelter where a transit flume is<br />

continuously supplied with wastewater from the sewer by means of a peristaltic pump. In this case, the<br />

representativeness of the wastewater in the flume and the varying position of the mobile pump intake<br />

in the sewer should be consi<strong>de</strong>red as additional sources of uncertainty. These were estimated to be<br />

approximately equal to 10 % of the measured value. Another example is related to the water level<br />

measurements. The sensor uncertainty of an ultrasonic probe is evaluated to be equal to 3 mm<br />

according to calibration experiments. As (i) the sensor is not perfectly installed and located in the<br />

sewer, and (ii) the free surface is not flat but experiences waves of height ranging from 1.5 cm in dry<br />

weather up to approximately 3 cm in w<strong>et</strong> weather events, the in situ standard uncertainty in the water<br />

level is s<strong>et</strong> equal to 7.5 mm - 15 mm <strong>de</strong>pending on the water level.<br />

Data correction<br />

This step comprises the correction of raw data at each time step according to the sensor calibration<br />

functions and the calculation of their standard uncertainties from the information provi<strong>de</strong>d in the<br />

calibration step.<br />

Corrected values Xˆ i of raw measurements X i at time step i are estimated by the inverse calibration<br />

function, either by direct analytical calculation (in case of calibration functions of <strong>de</strong>grees 1 and 2) or<br />

by numeric minimization (in less frequent case of calibration function of <strong>de</strong>gree 3), with addition of an<br />

offs<strong>et</strong> value when necessary.<br />

The total standard uncertainty in the corrected values is calculated by:<br />

u ˆ 2 ˆ<br />

tot Xi<br />

um<br />

Xi<br />

2 usiteX<br />

ˆ i 2<br />

where mX i <br />

uncertainties calculated according to the law of propagation of uncertainties (LPU), and <br />

(1)<br />

u ˆ is the measurement standard uncertainty accounting for sensor and calibration<br />

usite Xˆ i is<br />

the in situ standard uncertainty. The standard uncertainty and the LPU approach are applied according<br />

to international standards for the expression of uncertainty in measurements (ENV 1999, ISO, 2009).<br />

Selection of pre-validation tests and s<strong>et</strong>ting of test param<strong>et</strong>ers<br />

This step aims to select the tests to be applied (see Figure 1) and to s<strong>et</strong> the param<strong>et</strong>ers required for the<br />

pre-validation tests. The pre-validation m<strong>et</strong>hod is based on the m<strong>et</strong>hod initially <strong>de</strong>veloped by Mourad<br />

and Bertrand-Krajewski (2002). Three pre-validation marks are used: 1 (valid data), 2 (doubtful data)<br />

or 3 (incorrect data). Eight pre-validation tests are available and selected by the user according to the

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!