GPS-X Technical Reference

391 Optimizer
Note that the user can set the heteroscedasticity factors or have GPS-X estimate their
optimal values for you.
As process models are generally nonlinear, the parameter values that maximize
Equation 14.12 cannot be determined analytically. An iterative optimization method is
required to maximize Equation 14.12. As mentioned earlier in this chapter, GPS-X uses
the Nelder and Mead simplex method (Press et al., 1986) for optimization. This method
is a type of direct search algorithm that does not require the calculation of partial
derivatives. This is helpful when estimating parameters in systems of differential
equations. The simplex method also has the advantage that it can handle objective
functions containing discontinuities. This method is generally slower than derivativebased
optimization methods, but can handle a greater variety of objective functions and
is often found to be quite robust in finding solutions. The version of the simplex method
implemented in GPS-X allows for bounds to be placed on the parameters.
Due to the fact that the simplex method is designed for minimization, GPS-X minimizes
the negative of Equation 14.12 to determine the optimal parameter estimates.
Sum of Squares Objective Function
When there is only one target variable and the variance is constant across all observations
(i.e. the heteroscedasticity parameter is zero), the sum of squares objective function is
equivalent to the maximum likelihood objective function. For problems with more than
one target variable, the sum of squares objective function is a special case of the
maximum likelihood objective function that results if we make additional assumptions
about the measurement errors. Our maximum likelihood objective function, given by
Equation 14.12, is derived using the following assumptions:
The measurement errors are normally distributed random variables with a
mean of zero.
For each target variable, the measurement errors are independent from
observation to observation. Each target variable has its own variance which
varies from observation to observation according to a power-law. The
variances are unknown and are calculated as part of the optimization process.
There is no correlation between different target variables
If we make the additional assumptions that all of the variances are equal (i.e. all
responses have same variance) and the variances do not change from observation to
observation (i.e. the heteroscedasticities are all zero) then the maximum likelihood
function reduces to the sum of squares objective function in the multi-response case. The
assumptions used to derive the sum of squares objective function in the multi-response
case do not apply in most practical situations. Therefore, it is recommended that the
maximum likelihood objective function be used for calibration problems with more than
one target variable.
GPS-X Technical Reference