Software Engineering for Students A Programming Approach

29.4 Faults and reliability – estimating bugs 361
A valid complexity measure can potentially help software developers in the following
ways:
■ in estimating the effort needed in maintaining a component
■ in selecting the simplest design from amongst several candidates
■ in signaling when a component is too complex and therefore is in need of restructuring
or subdivision.
29.4 ● Faults and reliability – estimating bugs
The terminology adopted in this book is that a human error in developing software
causes a fault (a bug) which may then cause a failure of the system (or several different
failures). We have seen in Chapter 19 on testing that every significant piece of software
contains faults. Therefore if we are buying a piece of software it makes sense to
ask the supplier to tell us how many faults there are. If they respond by saying that
there are none, then they are either lying or incompetent. Similarly if we have developed
a piece of software, it would be professional (if we are honest) to be able to tell
users how many (estimated) faults there are and thus give the users some idea of the
expected reliability.
A commonly used metric for faults is fault density, which is the estimated number of
faults per 1,000 lines of code. Faults are detected both during verification and during
normal use after the software is put into productive use. Some faults are, of course, corrected
and therefore do not count towards the fault density. We must not forget that, in
addition to known faults, there are faults that are present but undetected. In commercially
written software, there is an enormous variation in fault densities – figures observed
are between 2 and 50 faults per KLOC (kilo lines of code). A figure of 2 is rated highly
creditable. The fault density metric is useful in assessing the effectiveness of verification
methods and as a measure of correctness (see below) in quality assurance.
Experimental studies suggest that most faults cause only rare failures, whereas a small
number of faults cause most of the failures. Thus it is more cost-effective to fix the small
number of faults which cause most of the trouble – if they can be found.
It would seem to be impossible to gauge how many faults remain in a thoroughly
tested system. After all if we knew what faults there are, we could correct them. One
technique arises from the earth sciences. How do you find out how many fish there are
in a lake? It would be costly (and kill many fish) to drain the lake. An alternative is to
insert additional, specially marked fish into the lake. These could be fish of a different
breed or slightly radioactive fish. After waiting a sufficient time for the fish to mix thoroughly,
we catch a number of fish. We measure the proportion of specially marked fish,
and, knowing the original number of special fish, scale up to find the total number of
fish. We can do the same thing in software by deliberately putting in artificial faults into
the software some time before testing is complete. By measuring the ratio of artificial
to real faults detected, we can calculate the number of remaining real faults. Clearly
this technique depends on the ability to create faults that are of a similar type to the
actual faults.