ETTC'2003 - SEE

Such an interdependence between tests and simulation allows a reuse throughout the
life of the system, by updating credible representation and authorizing successive risk levees.
By mastering reuse and validation processes, it is then possible to dispose of a powerful tool
in order to evaluate the system, including at its limit, as well as to explore new concepts and
predict performance, before launching any fabrication.
The following paragraphs will detail the virtuous circle between simulation and tests,
as well as the fundamental problem of validation.
However a few remarks are necessary to counter usual critics:
• “using simulation as support of a test is something we’ve always done”: true, but
what we discuss here goes much further than this, as we emphasize a necessary
coupling and a mutual benefit;
• “simulation will replace all tests”: this is not the intended goal, and is completely
foolish. The intended goal is to better conduct the physical tests that are strictly
necessary in order to acquire critical data for evaluation;
• “simulation can only reason on statistical data, and should not be trusted without
special care”: is it not the same when a series of tests is conducted in order to
induce a global performance, by considering only a few working points, which one
cannot always choose smartly to allow an authorized transition from local to global
behavior! Simulation, when interacting with tests, does not pretend to yield a
perfect result. By coupling both techniques, it should be possible to have a
consolidated credibility in the system to be developed.
On one hand, tests yield data from the real world, in given situations and
environments, therefore a priori credible, and they help to evaluate how technical
performance objectives and a given system maturity level are reached. During the tests,
security and environment preservation constraints have to be taken into account, which have a
heavy cost.
On the other hand, a simulation can be expensive, especially in order to guarantee the
validity of the developed models. It allows to predict experimentation results, by exploring all
possible solutions within a domain accessible to experimentation, and by extrapolating
performances outside the domain potentially accessible through tests.
Besides, some tests are easier to realize through simulation and much less expensive.
Indeed it is not necessary to concentrate, or move all systems to the same place: this
possibility of geographically distribution is a major advantage of simulation, in terms of
reactivity, delay and cost. For instance, it is much easier to evaluate different interoperability
tests (such as defined by NATO), without mobilizing as many resources. In the same way,
complex tests such as needed for systems-of-systems (e.g. extended air defense, or tactical
missile defense) become feasible, whereas they could not be performed before.
Therefore there is a true benefit to use tests and simulation in a cooperative manner. It
is also essential to promote simultaneously the reuse of validated models since there is then a
true multiplying effect in terms of risk and cost decrease.
Essential are the credibility of the information issued form M&S activities and the
identified limits of their use. They necessitate the definition and execution of an VV&A
(verification, validation and accreditation) process, as well as the acquisition of specific test
data, which will be used to validated simulations and models.
Indeed, using unreliable data and models could introduce supplementary risks within
an acquisition. This is very often used as an objection to a wider use of simulation. However
the same argument can be easily reversed: ill-referenced test data are also a risk factor, as they