Chapter 5 Robust Performance Tailoring with Tuning - SSL - MIT

Table 4.6: Tuning results on fifty hardware simulations: performance and number of
required tests.
# of Sims # Hardware tests
Method Success Failure Maximum Average
Iso Tuning 50 0 4 2.5
Hardware: BSD 50 0 231 46.2
Hardware: SA 50 0 2618 790
Worst-Case Model 29 21 2 2
Nominal Model 29 21 2 2
Optimized Model 24 26 2 2
AO Tuning 0 50 2 2
successful on only 50% of the trials. The only exception is AO tuning, which does not
successfully tune any of the simulations. It is worthwhile to note that, although the
model-only methods failed in the example discussed previously, there are some hard-
ware realizations for which these methods are adequate. Since each of these methods
require only two hardware tests, the initial test to assess nominal hardware perfor-
mance and a second one to try the tuning configuration, there is little cost in starting
the tuning process with these methods as much time and effort can be avoided if they
succeed. If the model-only method fails, the tuning configuration is a good starting
point for a hardware tuning scheme. Optimized model tuning and isoperformance
tuning are particularly compatible in this way since the uncertainty parameters used
for the model tuning also serve as an initial point on the isoperformance contour. It
is also interesting to note that the nominal model and worst-case model tuning meth-
ods result in the exact same numbers of successes. It turns out that these methods
succeed and fail on the same hardware realizations indicating that these methods are
interchangeable for this particular problem.
The hardware tuning methods and isoperformance tuning, on the other hand, are
consistently successful across the entire sample space. There is quite a difference in the
number of hardware tests that are required by the three methods, however. Real-time
simulated annealing is by far the least efficient requiring an average of 790 hardware
tests across the sample space. This result is not surprising since the method is a
random search of the tuning space. It is equivalent to simply trying random tuning
147