Recent developments in the Metamodel of Optimal ... - Dynardo GmbH

WOST 2011 • 24.– 25.11.2011 • © Dynardo GmbH 2011 

Recent developments in the 

Metamodel of Optimal Prognosis 

Thomas Most & Johannes Will 

Dynardo GmbH 

Recent developments in the Metamodel of Optimal Prognosis 


1


Introduction 

• Starting in 2008 the Metamodel of Optimal Prognosis has become an 

important feature in optiSLang’s sensitivity analysis, optimization and 

robustness analysis 

• The main priorities of the MOP methodology are: 

• Detection of the important variable subspace 

• Usage of understandable, reproducible and fast 

approximation models 

• Objective estimation of the approximation quality 

• However, some questions need to be discussed: 

• May more complex meta-models significantly improve the 

approximation quality? 

• How can we adapt the MOP to perform optimal in the framework 

of an optimization problem? 



2


Polynomial regression 

• Set of input variables 

• Definition of polynomial basis 

• Approximation function 

• Least squares solution 



3


Coefficient of Determination (CoD) 

• Fraction of explained variation 

of an approximated response 

• Total variation 

• Unexplained variation 

• Adjusted CoD 

to penalize over-fitting 



4


Limitations of the CoD 

• CoD is only based on how good the regression model fits through the 

sample points, but not on how good the prediction quality is 

• Approximation quality is too optimistic for small number of samples 

• For interpolation models with perfect fit, CoD is equal to one 

• Better approximation models are required for highly nonlinear 

problems, but CoD works only with polynomials 



5


Moving Least Squares (MLS) 

• Local polynomial regression with 

position-dependent coefficients 

• Distance depending weighting function 

• Smoothing of approximation is 

controlled by weighting radius D 

• Choice of D directly influences the CoD 



6


Coefficient of Prognosis (CoP) 

• Fraction of explained variation 

of prediction 

• Estimation of CoP by cross 

validation using a partitioning 

of available the samples 

• CoP increases with increasing number of samples 

• CoP is suitable for interpolation and regression models 

• With MLS continuous functions also including coupling terms 

can be represented with a certain number of samples 

• Prediction quality is better if unimportant variables are removed 

from the approximation model 



7


Metamodel of Optimal Prognosis (MOP) 

• Approximation of solver output by fast surrogate model 

• Reduction of input space to get best compromise between available 

information (samples) and model representation (number of inputs) 

• Advanced filter technology to obtain candidates of optimal subspace 

• Determination of optimal approximation model (polynomials, MLS, …) 

• Assessment of approximation quality (CoP) 

MOP solves 3 important tasks: 

• Best variable subspace 

• Best meta-model 

• Estimation of prediction quality 



8


Example: Analytical nonlinear function 

• Additive linear and nonlinear terms and one coupling term 

• Contribution to the output variance (reference values): 

X 1 : 18.0%, X 2 : 30.6%, X 3 : 64.3%, X 4 : 0.7%, X 5 : 0.2% 



9



MOP vs. Polynomials & MLS 

• 100 LHS support points 

• Explained variation of independent test data 

set with 100 samples as error measure 

• With increasing number of unimportant variables the approximation 

error increases for polynomials and MLS 

• MOP detects 3 important variables 



10



MOP vs. Kriging 

• Kriging optimized with maximum likelihood will not always give the 

optimal correlation parameters 

• For certain number of inputs Kriging seems to be more suitable than 

Moving Least Squares 



11



MOP vs. Support Vector Regression & Artificial Neural Networks 

• SVR and ANN require much more effort for training 

• ANN approximation quality strongly decreases with increasing dimension 

• Kriging approximation seems to be similar or even more accurate 



12



MOP vs. MATLAB’s stepwise regression 

• Stepwise regression selects coefficients of polynomial regression by 

statistical significance testing 

• For a larger number of inputs F-test based MATLAB implementation 

performs only well if linear terms are considered 



13


New implementation of MOP 

• Implementation based on state-of-the-art algebra library 

• Parallel computing will be introduced at several levels (research project in 

cooperation with Fraunhofer ITWM founded by KMU-innovativ/BMBF) 

• Efficient CoP implementation enables very fast testing 

– Analytical estimator for polynomials with Box-Cox 

– No extra computational effort for MLS 

• New filter technology for the detection of interaction terms between 

input variables in high dimensional problems 

• Hierarchical model testing to reduce overall computational effort and to 

enable more time consuming models such as Kriging 



14


Sensitivity analysis vs. optimization using MOP 

• MOP is build by equally weighting the prediction errors: 

• This is optimal for sensitivity analysis since global variance-based 

measures (single variable CoPs) are most accurate with equal weights 

• Optimization requires good approximation quality of objective and 

constraint functions in the region around the optimum 

� Weighting of prediction errors to improve the MOP based optimization 

• Extrapolation of approximation function often looses accuracy 

� Additional constraint to consider approximation quality 



15


Example: Rosenbrock function 

Optimization using MOP 

• Optimum at 1.0;1.0 

• Best design from 500 LHS 

samples at 0.5;0.3 

• Global CoP is 99.996% 

• Optimum found on MOP 

approximation at 1.7;3.0 

• Approximation is globally very 

good but locally not sufficient: 

� Weighted CoP is 73.24% 

(with the samples f≤20) 

� Choice of optimal influence 

radius needs to consider 

local/weighted CoP 



16


Example: Rosenbrock function 

Optimization using Kriging 

• Global CoP is 99.999% 

• Optimum found on Kriging 

approximation at 1.1;1.2 

• Weighted CoP is 99.912% 

(with the samples f≤20) 



17


Conclusions 

• CoP based detection of important variables enables application of MOP 

for high-dimensional industrial problems 

• Using the optimal subspace is often more promising than using 

highly complex approximation models 

• New implementation will significantly accelerate MOP approach and 

enables the consideration of more time consuming models (Kriging) 

• New filter technology will improve the variable detection 

• Optimization requires special strategies 

• Weighted CoP to get best approximation around the optimum 

• Consideration of local approximation quality in optimization method 



18

Recent developments in the Metamodel of Optimal ... - Dynardo GmbH

Create successful ePaper yourself

Delete template?

Save as template?