Software Engineering for Students A Programming Approach

386 Chapter 31 ■ Assessing methods
31.2 ● How to assess methods
Is it possible to identify a collection of tools and methods that are ideal in all circumstances?
The answer is no. Software engineering is at an exciting time. There are a dozen schools
of thought competing to demonstrate their supremacy and no single package of tools and
methods seems set to succeed. Some methods seem particularly successful in specific areas,
for example, the data structure design method in information systems. Other methods, like
structured walkthroughs, seem generally useful. In the field of programming languages,
declarative languages have become important in expert systems, while highly modular
imperative languages are widely used in real-time and command and control systems.
Ideally, metrics (Chapter 29) would enable us to determine the best method or combination
of software development methods. Regrettably, this is virtually impossible. The
first problem is identifying the criteria for a best method. As we saw in Chapter 1 on
problems and prospects, there are usually a number of goals for any software development
project. In order to choose between methods it is necessary to establish what
blend of criteria is appropriate for the particular project. For example, the set of goals
for a particular project might be to optimize:
■ development effort,
■ reliability, and
■ maintainability
and these are in conflict with each other. In general, the most significant conflict is
probably between development effort and reliability of the product. For example, a
safety-critical system needs to be highly reliable. However, for a one-off program for a
user to extract information from a database, the prime goal may be quick delivery.
There can be no set of factors that allow universal comparison between methods.
Equally, it is unlikely that there will ever be a single best method.
Suppose that we had narrowed down the choice to two applicable methods, called
A and B. What we would like to have is hard evidence like this: “Method A gives 25%
better productivity than method B.” Regrettably, there is no such data available today,
because of the enormous difficulty of creating it. Let us examine some of those difficulties.
Because of cost, it is virtually impossible to conduct any realistic experiments in
which two or more methods are compared. (The cost of developing the same piece of
software twice is usually prohibitive.) Usually the only experimental evidence is based
on scaled-down experiments. Suppose, for example, that we wanted to compare two
design methods, A and B. We could give ten people the specification of a small system
and ask them to use method A, and similarly we could ask a second group to use
method B. We could measure the average time taken to complete the designs and hence
hope to compare the productivities of the methods. We could go on to assign additional
problems and employ more people to increase our confidence in the results. Ultimately,
we might gain some confidence about the relative productivity of the two methods.
But many criticisms can be aimed at experiments like these. Are the backgrounds of
the participants equal? Is the experience of the participants typical? (Often students are
used in experiments, because they are cheap and plentifully available. But are students
typical of professional software developers?) Have sufficient number of people taken