Software Engineering for Students A Programming Approach
Software Engineering for Students A Programming Approach Software Engineering for Students A Programming Approach
Figure 19.2 Unit testing Driver Component under test 19.8 System (integration) testing 277 Thus the component under test is surrounded by scaffolding. This is a large undertaking. In many developments, the collections of drivers and stubs is often as big as the software itself. 19.8 ● System (integration) testing Thus far we have only considered unit testing – testing an individual software component, a method or a class. We have implicitly assumed that such a component is fairly small. This is the first step in the verification of software systems, which typically consist of tens or hundreds of individual components. The task of testing complete systems is called system or integration testing. Suppose that we have designed and code all the components for a system. How can we test these components and how can we test the complete system? Here are three different approaches to system testing: 1. big bang – bring all the components together, without prior testing, and test the complete system 2. improved big bang – test each component individually (unit testing), bring them all together and test the complete system 3. incremental – build the system piece by piece, testing the partial system at each stage. The first approach – big bang or monolithic testing – is a recipe for disaster. There is no easy way of knowing which component is the cause of a fault, and there is an enormous debugging task. The second approach is slightly better because when the components are brought together, we have some confidence in them individually. Now any faults are likely to be caused by the interactions between the components. Here again, there is a major problem of locating faults. An alternative is to use some form of incremental testing. In this approach, first one component of the system is tested, then a second component is linked with the first and the system tested. Any fault is likely to be localized either in the newly incorporated component or in the interface between the two. We continue like this, adding just one component at a time. At each stage, any fault that presents itself is likely to be caused by the new component, or by its interface to the system. Thus fault finding is Stub Stub Stub
278 Chapter 19 ■ Testing made considerably easier. Various approaches are explained in Chapter 24 on incremental development. 19.9 ● Discussion We have seen that exhaustive testing is infeasible. Therefore complete testing is impossible and, whatever testing methods are used, they can never ensure that the software is free from bugs. Thus testing is a poor technique but until formal verification becomes widely applicable it is a vital technique. However much we test our programs, using all our skill and intuition, we can never be sure that we have eradicated all the faults. The situation is well summed up by one of computing’s gurus, Dijkstra, in his famous remark, “Testing can only show the presence of bugs, never their absence.” This has been (anonymously) rephrased as, “Just because you have never seen a mermaid doesn’t mean that they don’t exist.” It can be reassuring to adopt the view that a test that reveals no bugs is a successful test. But rather we should look upon such a test as unsuccessful! It is difficult to get accurate data on the number of bugs present in production software because, unsurprisingly, organizations do not want to reveal this kind of information. The indications are that there are typically between 2 and 50 bugs per 1,000 lines of source code in commercial production software. A figure like this is more properly called a fault density. It measures the number of known faults per 1,000 lines of code (LOC). A figure of 2 is considered to be most creditable. Ways of measuring this quantity are explained in Chapter 29 on metrics and quality assurance. The trouble is, of course, that bugs always surface at the worst possible time, for example, when you are demonstrating the completed software to the client. This phenomenon has long been known to students of reliability, who quote Murphy’s laws: 1. “If a system can fail, it will,” 2. “and at the worst possible moment.” Another, more objective, observation is that some bugs create serious faults, while others lie dormant and do not give any trouble. Chapter 31 on assessing methods looks at the evidence that is available to compare verification techniques, including testing. The surprising indications are that simply inspecting code is more effective than carrying out testing. The worrying conclusion to any discussion of verification is that all software (of any significant size) contains faults. Summary Testing is one set of techniques for verifying software. Exhaustive testing is a practical impossibility.
- Page 249 and 250: 226 Chapter 16 ■ Programming in t
- Page 251 and 252: 228 Chapter 16 ■ Programming in t
- Page 253 and 254: 230 Chapter 16 ■ Programming in t
- Page 255 and 256: 232 Chapter 16 ■ Programming in t
- Page 257 and 258: 234 Chapter 16 ■ Programming in t
- Page 259 and 260: 236 Chapter 16 ■ Programming in t
- Page 261 and 262: 238 Chapter 17 ■ Software robustn
- Page 263 and 264: 240 Chapter 17 ■ Software robustn
- Page 265 and 266: 242 Chapter 17 ■ Software robustn
- Page 267 and 268: 244 Chapter 17 ■ Software robustn
- Page 269 and 270: 246 Chapter 17 ■ Software robustn
- Page 271 and 272: 248 Chapter 17 ■ Software robustn
- Page 273 and 274: 250 Chapter 17 ■ Software robustn
- Page 275 and 276: 252 Chapter 17 ■ Software robustn
- Page 277 and 278: 254 Chapter 17 ■ Software robustn
- Page 279 and 280: 256 Chapter 17 ■ Software robustn
- Page 281 and 282: 258 Chapter 17 ■ Software robustn
- Page 283 and 284: 260 Chapter 18 ■ Scripting GNU/Li
- Page 285 and 286: 262 Chapter 18 ■ Scripting In sum
- Page 288: PART D VERIFICATION
- Page 291 and 292: 268 Chapter 19 ■ Testing We begin
- Page 293 and 294: 270 Chapter 19 ■ Testing within a
- Page 295 and 296: 272 Chapter 19 ■ Testing Test num
- Page 297 and 298: 274 Chapter 19 ■ Testing if (a >=
- Page 299: 276 Chapter 19 ■ Testing 3. apply
- Page 303 and 304: 280 Chapter 19 ■ Testing 19.3 Dev
- Page 305 and 306: 282 Chapter 19 ■ Testing 19.2 The
- Page 307 and 308: 284 Chapter 20 ■ Groups The term
- Page 309 and 310: 286 Chapter 20 ■ Groups Of course
- Page 311 and 312: 288 Chapter 20 ■ Groups • Exerc
- Page 314 and 315: CHAPTER 21 This chapter explains: 2
- Page 316 and 317: Stage Input Output 21.3 Feedback be
- Page 318 and 319: Summary The essence and the strengt
- Page 320 and 321: CHAPTER 22 This chapter: 22.1 ● I
- Page 322 and 323: 22.2 The spiral model 299 to try to
- Page 324 and 325: 22.4 ● Discussion Exercises 301 A
- Page 326 and 327: CHAPTER 23 Prototyping This chapter
- Page 328 and 329: Therefore, in summary: ■ the prod
- Page 330 and 331: 23.5 Evolutionary prototyping 307 U
- Page 332 and 333: Reuse components 23.6 Rapid prototy
- Page 334 and 335: Pitfalls For users, the problems of
- Page 336 and 337: Answers to self-test questions 313
- Page 338 and 339: 24.2 ● Big-bang implementation 24
- Page 340 and 341: Tested component Figure 24.1 Top-do
- Page 342 and 343: 24.7 ● Use case driven implementa
- Page 344 and 345: ■ middle-out ■ use case based.
- Page 346 and 347: SELF-TEST QUESTION 25.1 What is the
- Page 348 and 349: sharing of software or their own re
Figure 19.2 Unit testing<br />
Driver<br />
Component<br />
under test<br />
19.8 System (integration) testing 277<br />
Thus the component under test is surrounded by scaffolding. This is a large undertaking.<br />
In many developments, the collections of drivers and stubs is often as big as the<br />
software itself.<br />
19.8 ● System (integration) testing<br />
Thus far we have only considered unit testing – testing an individual software component,<br />
a method or a class. We have implicitly assumed that such a component is fairly<br />
small. This is the first step in the verification of software systems, which typically consist<br />
of tens or hundreds of individual components. The task of testing complete systems<br />
is called system or integration testing.<br />
Suppose that we have designed and code all the components <strong>for</strong> a system. How can<br />
we test these components and how can we test the complete system?<br />
Here are three different approaches to system testing:<br />
1. big bang – bring all the components together, without prior testing, and test the<br />
complete system<br />
2. improved big bang – test each component individually (unit testing), bring them<br />
all together and test the complete system<br />
3. incremental – build the system piece by piece, testing the partial system at each<br />
stage.<br />
The first approach – big bang or monolithic testing – is a recipe <strong>for</strong> disaster. There is<br />
no easy way of knowing which component is the cause of a fault, and there is an enormous<br />
debugging task. The second approach is slightly better because when the components<br />
are brought together, we have some confidence in them individually. Now any<br />
faults are likely to be caused by the interactions between the components. Here again,<br />
there is a major problem of locating faults.<br />
An alternative is to use some <strong>for</strong>m of incremental testing. In this approach, first one<br />
component of the system is tested, then a second component is linked with the first<br />
and the system tested. Any fault is likely to be localized either in the newly incorporated<br />
component or in the interface between the two. We continue like this, adding<br />
just one component at a time. At each stage, any fault that presents itself is likely to be<br />
caused by the new component, or by its interface to the system. Thus fault finding is<br />
Stub<br />
Stub<br />
Stub