Downloadable - About University

Assessment of the validity of probability forecasts 287
occurrence may psychologically disassociate an event from its event
set. As we saw earlier in Chapter 9, a similar phenomenon has been
notedbyTverskyandKahneman 14 and the term ‘representativeness’
was coined to refer to the dominance of individuating information in
intuitive prediction.
Clearly, judgmental forecasts should be monitored for additivity and
incoherence should be resolved. However, a simple normalization may
not be a quick and easy solution to incoherence. Lindley et al. 15 outlined
a major problem:
Suppose that I assess the probabilities of a set of mutually exclusive and
exhaustive events to be
0.001, 0.250, 0.200, 0.100, 0.279
It is then pointed out to me that these probabilities sum to 0.830 and
hence that the assessment is incoherent. If we use the method ... with the
probability metric, we have to adjust the probabilities by adding 0.034 to each
(= (1/5)(1 − 0.830)) to give
0.035, 0.284, 0.234, 0.134, 0.313
The problem is with the first event, which I originally regarded as very
unlikely, has had its probability increased by a factor of 35! Though still small
it is no longer smaller than the others by two orders of magnitude.
Obviously, other methods of allocating probability shortfalls can be
devised, but our view is that the best solution to such problems is for the
decision analyst to show the decision maker his or her incoherence and
so allow iterative resolution of departures from this (and other) axioms
of probability theory. Such iteration can involve the analyst plotting the
responses on a graph (e.g. as a cumulative distribution function) and
establishing whether the decision maker is happy that this is an accurate
reflection of his or her judgments. Finally, the decision maker can be
offered a series of pairs of bets. Each pair can be formulated so that the
respondent would be indifferent between them if he or she is behaving
consistently with assessments which were made earlier.
Assessment of the validity of probability forecasts
A major measure of the validity of subjective probability forecasts is
known as calibration. By calibration we mean the extent to which the