Analysis Techniques For Man-Machine Systems Design

NATO UNCLASSIFIED- 41 - AC/243(Panel-8)TR/7Volume 1* schedules which show that the analyses will be timely* organization charts which indicate that the human engineering effort will be integratedwith other systems engineering and Integrated Logistics Support (ILS) activities* use of metrics and measures of effectiveness which are compatible with each other andwith other engineering activities* compliance with a relevant specification (see Chapter 5: there are few specificationswhich are relevant to the QA of human engineering analyses)71. Some of these items are addressed in the volume on Quality Assurance in the BritishHuman Factors Guidelines for the design of Computer-Based Systems (Ministry of Defence.1988). However, most of the criteria listed in that publication are subjective (e.g., "Are theproposed design teams satisfactory? If not, why not?'). Some of the above evidence is coveredin the exhaustive checklists contained in a report on human factors in system acquisitionproduced for the U.S. Navy (Malone et al., 1986). The entries under "Quality AssuranceConsiderations" in the reviews of individual analysis techniques (see Volume 2) were based onexperience and on comments from users. They show a general pattern ofreference to"completeness," "consistency" (with either the statement of requirements (SOR), precedinganalyses, or internally), and "accuracy" (e.g., of task time estimates). "Review by experts" isalso referred to but does not represent a criterion because. presumably, the "experts" have theirown criteria for judging the analyses. Figure 3.3 shows the ranking of the QA criteria reported inVolume 2. The criteria average 1.4 checks for each analysis technique reviewed (36 checks for26 techniques).CompletenessConsistency wth preceding analysisInternal consistencyReview by expertsAccuracy (e.g., of time estimates)Computer checkingVerification by subsequent analysesSensitivity analysis0 2 4 6 8Figure 3.3: Number of quality assurance criteria used forhuman engineering analyses reviewed In Volume 272. None of the entries on the techniques reviewed in Volume 2 refers to checking theanalyses for timeliness, or for compatibility with other systems engineering or ILS activities. Noentry refers to checking the analysis against a specification for such analyses. Suchconsiderations should be included in the development of the plan for the human engineeringanalyses. The project manager and/or the analyst should employ the following QA criteria:* completeness* consistency* timeliness* compatibility with other engineering analvses