Downloadable - About University

Aggregating judgments in general 313
example, suppose that three individuals, Allen, Bailey and Crossman,
make the following estimates of the cost of launching a new product:
$5 million, $2.5 million and $3 million. We think that Allen is the best
judge and Crossman the worst, and we therefore decide to attach weights
of 0.6, 0.3 and 0.1 to their estimates (note that if the weights do not sum
to one then this can be achieved by normalizing them – see Chapter 3).
The group estimates will therefore be:
(0.6 × $5m) + (0.3 × $2.5m) + (0.1 × $3m) = $4.05m
Clearly, the main problem of using weighted averages is that the
judgmental skills of the group members need to be assessed in order to
obtain the weights. Methods which have been proposed fall into three
categories: self-rating, rating of each individual by the whole group
(see, for example, De Groot 3 ) and rating based on past performance.
However, there can be difficulties in applying these methods. The
first two approaches compound the individual’s judgmental task by
requiring not only judgments about the problem in hand but also those
about the skill of individual group members. In some circumstances these
problems can be avoided by using weights based on past performance,
but as Lock 4 points out, even here there can be difficulties. The current
judgmental task may not be the same as those in the past. For example,
the quantity which an individual has to judge may be less familiar
than those which have been estimated previously. Furthermore, past
performance may be a poor guide where judges have improved their
performance through learning.
Clearly, simple averaging avoids all these problems, so is it worth
going to the trouble of assessing weights? Research in this area has
consistently indicated that simple averages produce estimates which are
either as good as, or only slightly inferior to, weighted averages (see, for
example, Ashton and Ashton 2 ). Ferrell 1 suggests a number of reasons for
this. He argues that groups tend to be made up of individuals who have
very similar levels of expertise and access to the same information. In
the case of small groups, even if we are fortunate enough to identify the
best individual estimate, its accuracy is unlikely to be much better than
that of the simple average of the entire group’s judgments. Ferrell also
points out that few experiments provide the conditions where weighting
would be likely to offer advantages. He suggests that these conditions
exist when there is:
a moderately large group of well-acquainted individuals that frequently
works together and has a wide range of different types of expertise to bring
to bear on questions that require an equally wide range of knowledge.