MIL-STD-2165 - Barringer and Associates, Inc.

.MIL-STD-216526 JANUARY 1985MILITARYSTANDARDTESTABILITY PROGRAMFOR ELECTRONIC SYSTEMSAND EQUIPMENTS●“oAMSCNo. N3423 ATTS

MIL-STD-2165DEPARTMENT OF DEFENSEWASNINGTON, DC 20301Testability Program for Electronic Systems and EquipmentsMIL-STD-21651. This Military Standard is approved foruaebyalf Departments and Agencies of the *Department of Defense.2. Beneficial comments (recommendations, additions, deletions) and any pertinentdata which may be of use in improving this document should be addressed to: ‘Commander, Naval Electronic Systems Command (ELEX-6111), Washington, DC 20363-5100, by using the self-addressed standardization Document Improvement Proposal (DDForm 1426) appearing at the end of this document or by letter.●9,ii

MIL-STD-2165●FOREWORD1. Testability addresses the extent to which a system or unit supporta fault detectionand fault isolation in a confident, timely and cost-effective manner. The incorporationof adequate testability, including built-in test (BIT), requires early end systematicmanagement attention to testability requirements, design and measurement.2. This standard prescribes a uniform approach to testability program planning,establishment of teatability (including BIT) requirements, testability analysis, prediction, and evaluation, and preparation of testability documentation. Included ere:.a.b.:e.f.g.h.Testability program planningTestability requirementsTestabiti ty designTestability predictionTestability demonstrationTestability data collect ion end analysisDocumentation of testability programTestability review.3. This standard also prescribes the integration of these testability programrequirements with other closely related, interdisciplinary program requirements, such asdesign engineering, maintainability and logistic support.4. Three appendices are included to augment the teslrsof this standard:oa. Appendix A provides guidance in the selection and application of testabilitytasks.b. Appendix B describes the Inherent Testability Assessment which provides ameasure of testability earlyin the design phase.c. Appendix C provides a Glossary of terms used in this standard.*.iii

MfL-STD-2165CONTEN13Paragraph 1.1.11.21.32.2.1SCOPEPurposeApplicationTailoring of TeakaREFERENCED DOCUMENTSfssues of Documents111111.3.3.13.2DEFINITIONS AND ACRONYMSDefinitionsAcronyms and abbreviations1114.4.14,24.3GENERAL REQUIREMEN~Scope of testability programTestability program requirementsApplication of requirements22235.5.15.2DETAILED REQUIREMENTSTeak descriptionsTask integration3336.6.1NOTESData requirements33●‘!TASN SECTIONSTask 100.101.102.1030PROGRAM MONITORING AND CONTROLTestability program planningTestability reviewsTestability data collection and analysis planning5679200.201.202.203.DESIGN AND ANALYSISTestability requirementsTestabilityy preliminary design and analysisTestability detail design and analysis10111315*300.301.TEST AND EVALUATIONTestability inputs to maintainabilitydemonstration1718APPENDICESAppendixABcTestability Program Application GuidanceInherent Testability AssessmentGlossary206073iv

MIL-STD-21653.2 Acronyms and abbreviations. The following acronyms end abbreviations listed inthis ?vi>litaryStandard are defined es follows:a.b.c.d.e.f.&h.i.j.k.1.m.n.o.e.q.r.s.t.u.v.w.x.Y.z.ea.bb.zCC.ff.m.Ml.ATE - automatic taut equipmentATLAS - abbreviatedBIT - built-in testtest language for all systemsBITE - built-in test equipmentCDR - critical design reviewCDRL - contract data requirements liitCFE - contractor furnished equipmentCI - configuration itemCND - cannot duplicateDID - data item descriptionD&V - demonstration and validationFMEA - failure modes and ef fecte analysisFQR - formal quti]fication reviewFSD - full-scale developmentGFE - government furnished equipmentGPETE - general purpose electronic test equipmentHITS - hierarchical interactive test simulatorID - interface deviceI/O - input or outputLSAR - logistic support analysis recordILSMT - integratedLSA - logistic supportlogistic support managementanalysisteamMTTR - mean time to repairP/D - production end deploymentPDR - preliminary design reviewR&M - reliability end maintainabilityROM - read only memorySCOAP - sandia controllabtiltySDR - system design reviewobservability anfdysis programSTAMP - system testability and maintenance programT&E - test and evaluationTPS - test program setTRD - test requirementsUUT - unit under testdocument4. GENERAL REQUIREMENTS #4.1 scope of testebtity prOgram. This standard is intended to impose and facilitateinter-iisciplurary efforts requmecf to develop testable systems and equipments. The ,testability program scope includes:a. Support of end integration with maintainability design, includingrequirements for performance monitoring and corrective maintenance actionat alf levels of maintenance.b. Support of integrated logistic support requirements, including the support andtest equipment element and other logistic elements.●2●

..- — . . . .MISJ_S1-SJ_610~●c. Support of and integration with design engineering requirements, includingthe hierarchical development of testability designs from the piece part to thesystem.4.2 Tcstabiity prc@e m requ irements. A testabiii ty program shetl be estabtiihed whichaccomplishes the following general requirements:.a. preparation of a Testability Program Planb. Establishment of sufficient, achievable, and affordable testability, built-inand off-line test requirementsc. Integration of testability into equipments and systems during the designprocess in coordination with maintainability design processd. Evaluation of the extent to which the design meets testability requirementse. Inclusion of testability in the program review process.4.3 Application of reqra“remente. Detailed requirements described in this standard areto be selectively applied and are intended to be tailored, es required, and es appropriateto particular systems and equipment acquisition programs. Appendix A providesrationale end guidance for the selection and taitoring of testability program taska.5. DETAILED REQUtREMEN’IS●S.1 ‘lWskd~riptiors. Individual task requirements are provided for the establishmentof a testabdlty program for electronic system and equipment acquisition. The tasks arecategorized es foUows:TASK SECTION 100.PROGRAhl h!ONfTORING AND CONTROLTask 101 Testability Program PlanningTeak 102 Testability ReviewsTask 103 Testability Data Collection and Analysis PlanningTASK SECTtON 200.DESIGN AND ANALYSISTask 201Task 202TestabilityTestabilityRequirementsPreliminary Design and AnalysisTask 203 Testability Detail Design and Anatysis.TASK SECTION 300. TEST AND EVALUATIONTask 301 Testability Inputs to Maintainability DemonstrationsThe individual task requirements provide for integration withot%%%RP@’ engineering and management tasks to preclude duplication and overtepwhile assuring timely consideration and accomplishment of testability requirements.6. NOTES06.1 Datfr requirements. When this standard is used in an acquisition, the dataidenti~kd below shall be deliverable only when specified on the DD Form 1423 Contract3

MIL-STW2165Data Requirement LMt (CDRL). When the DD Form 1423 is not used and DefenseAcquisition Regulation 7-104. 9(n) (2) is cited, the data identified below shelf be deliveredin accordance with requirements sDecified in the contract or Durchase order. ●Deliverable data esaocia~ed with the ‘requirements of this standard are cited in thefollowing teaks:TaskData RequirementApplicable Data ItemDescription (DID)101 Testability Program Plan102 Program Review DocumentationDI-T-7198DI-E-5423*103 Data Collection and Analysis Plan DI-R-7105201, 202, 203 Testabilityy Analysis Report DI-T-7199301 Maintainability DemonstrationTest PlanDI-R-7112Maintainability Demonstration DI-R-7113● Equivalent approved DID may be used.(Copies of DIDs required by contractors in connection with specific acquisition functionsshould be obtained from the Naval Publicetiom and Forms Center or es directed by thecontracting of ficer.)●Custodians: Preparing Activity:Army CR Navy-ECNavy ECAir Force 17 (Project ATTS-0007)Review:User:.4

-—_. —- —●✎✎TASK SECTION 100PROOEAM MONITOEING AND CONTROL0.5

MIL-STD-2165&h.i.j.k.Lm.n.Review coordination between BIT hardware and BIT software efforts.Review BIT interface to operator and maintenance personnel.Review BIT fault detection and fault isolation measures to be USOCL●Identify models used and model assumptions. Identify any methods tobe used for automatic test generation and test grading.Review BIT fault detection and fault isolation performance todetermine if BIT specifications are met. Review efforts to improve BITperformance through improved tests or item redesign. Assess adequacy .of testability /maintainability data interfaces.Review testability parametem to be included in Maintainability .Demonstration. Identify procedures through which testability concernsare included in Demonstration Plans and Procedures.Review compatibility of signal characteristics at selected test pointswith planned teat equipment. Assess adequacy of data interfacebetween testability and Support and Test Equipment organizational !elements.Review performance monitoring, BIT design and off-Iine testrequirements to determine completeness and consistency.Review approaches to monitoring production testing and fieldmaintenance actions to determine fault detection and fault isolationeffectiveness.Review plans for evaluating impact on testability for Engineering ●Change Proposafs.102.3 TASK INPUT102.3.1 Identification of amount of time to be devoted to the testability program ateach formal review and the level of technical detail to be provided. *102.3.2 Identification of level of participation desired by the requiring authority ininternal and subcontractor testability design reviews. *102.4 TASK OUTPUT102.4.1 Document ed results of testabi~ ty assessment as an integral part of systemprogram review documentation. (102.2.1)102.4.2 Documented results of testability design reviews, including action itemspending. (102.2.2)“1-1I~specifiedby the requiring authority.8

●mABILtTYMIL-WD-2165TASK 103DATA COLLECTION AND ANALYSIS PLANNING103.1 PURPOSE. To estabfish a method for identifying and tracking te-stabitityretatedproblems during system production and deployment and identifying correctiveact ions.103.2 ‘TASKDESCFtIPTtON103.2.1 Develop a plan for the analysis of production test resutte to determine if BIThardware and software, ATE hardware and software, and maintenance documentation aremeeting specifications in terms of fault detection, fault resolution, fault detection timesand fault isolation times.103.2.2 Develop a plan for the analysis of maintenance actiom for the fielded systemto determine if BIT hardware and software, ATE hardware and software, andmaintenance documentation are meeting specifications in terms of fault detection, faultresolution, false indications, fault detection times and fautt isolation times.103.2.3 Define data collection requirements to meet the needs of the testabilityanatysis. ‘t%edata cotlected shaU include a description of relevant operational anomahsand maintenance actions. Data coUection shaU be integrated with similar datacottection procedures, such as those for reliabltity. and maintainabtiity, and LogiiticSupport Anatysis and shaU be compatible with specified data systems in use by themititary user organization.o103.3 TASK INPUT103.3.1 Identification of field or depot test eauipment (either government furnishedequipment or contractor furnished @ipment) ‘to- be available- for production anddeployment testing. ”,103.3.2 Identification of existing data coUection systems in use by the usingcommand. *103.3.3 Relationship of Task 103 to Task 104 of MIL-STD-785 and Task 104 of MIL-STD-470. *. 103.4 TASK OUTPUT103.4.1 Testability data coUection and analysis plan for production test; documentedin accordance with DI-R-7105. (103.2.1)103.4.2 Testability data coUection end anstysis ptan for analyzing maintenanceactions on fielded systems; documented in accordance with D1-R-7105. (103.2.2 and103.2.3).●To be specifiedby the requiring authority.9

MIL-W’D-2165oTASK SECTION 200DE91GN AND ANALYSIS●10●

MU-STD-2165TASK 201Tl?S1’ABILITYREQUIREMI.WR2I 201.1 PURPOSE. To (t) recommend system test and testability y requirements whichbest achieve availability end supportability requirements end (2) allocate thoseI requirements to subsystems and items.201.2 TASK DWWtIPTION“201.2.1 EWablish overall testability de-sign objectives, god% thresholds endconstraints in support of the logistic support analysis process of MIL-STD-1388-1 or.equivalent supportability anetysis approved by the requiring authority. In this analysis,prime system design for testability is to be one of the elements to be traded off forireproved supportabllit y. Inputs to these requirements include:a. Identification of technology advancements which can be exploited insystem development and test development and which have the potential for increasingtesting effectiveness, reducing test equipment requirements, reducing test costs, orenhancing system availability.b. Identification of existing and ptanned test resources (e.g., family oftesters in inventory) which have potential benefits. Identify tester limitations.c. ldentific8tion of testing end testability problems on similar systemswhlch should be avoided.0201.2.2 Establish performance monitoring, built-in test and off-line test objectivesfor the new system at the system and subsystem levets. identify the risks anduncertainties in;olved in achieving the objectiv=- established.201.2.3 Estabtish BIT, test equipment and testabitit y constraints for the new system,such as limitations on additional hardware for BIT, for inclusion in system specificationsor other requirement documents. These constraints shell incbsde both quantitative andqualitative constraints.201.2.4 Evetuate alternative diagnostic concepts to include varying degrees of BIT,manual end off-line automatic testing, diagnostic test points, etc., and identify theselected diagnostic concept. The evaluation shell include:a. A determination of the sensitivity of system readiness parameters tovariations in key testability parameters. ‘l?sese parameters include BIT fault detection,fatse atarm rate, etc.b. A determination of the sensitivity of life cycle costs to variations inkey testability parameters.c. An estimation of the manpower and personnel implications ofalternative diagnostic concepts in terms of direct maintenance manhours per operatinghour, job classifications, skill levels, and experience required at each level ofmaintenance.11

IMIL.+TD-2165 1d. An estimation of risk associated with each concept.1●201.2.5 Establish BIT performance requirements at the system and subsystem level.These requirements include specific numeric performance requirements imposed by therequiring authority. Other requirements shell be based, in pert, oma. Maximum allowable time between the occurrence of a failure conditionand the detection of the failure for each mission function.b. Maximum allowable occurrence of system downtime due to erroneous .failure indications (BIT false alarms).c. Maximum allowable system downtime due to corrective maintenanceactions at the organizational level.d. Minimum life-cycle costs.201.2.6 Recommend BIT and testability requirements for inclusion in systemspecifications. Appendix A, Figure 5, provides guidance on requirements to be specified.201.2.7 Allocate BIT and testability requirements to configuration itemspecifications baaed upon reliability and criticality considerations.201.3 TASK INPUT201.3.1 Supportability analysis data in accordance with MIL-STO-1386-I or othermethod approved by the requiring authority.201.3.2 Reliability and maintainability analysis and requirements such es from Task●203 of MIL-STD-785 and Task 205 of MIL-STD-470.201.3.3 Specific numeric BIT and testability requirements.;201.4 TASK OUTPUT201.4.1 Testability data required for supportability analysis. (201.2.1 through201.2.4)201.4.2 Description of selected diaf?nostic concept and tradeoff methodology,evaluation criteri~, models used, and anal~sis results; doc_umented in accordance with fi_I- .T- 7199. (201.2.4)201.4.3 Recommended BIT and testability y requirements for system specification.(201.2.3, 201.2.5 and 201.2.6)201.4.4 Recommended BIT and testability requirements for each configuration itemspecification. (201.2.7)~To be specifiedby the requiring authority.120

MIL%l’D-2165●-TABK 202‘TESTABILITYPRRLtMfNARY D~GNANO ANALYSIB202.1 PURPOBE. To incorporate testability design practices into the design of asystem or equipment early in the dmign phase and to assess the extent to whichtestability is incorporated.202.2 TASK DEBCRIPITON202.2.1 Institute teatebilty design concepts as an integrat part of the system orequipment design process. Guidance is provided in Appendix A, paregraph 50.6, on theuse of these concepts.202.2.2 incorporate appropriate te=stabitity design concepts into the preliminarydesign for each item. Develop BIT hardware, software end firmware as en integrat partof the design effort.●202.2.3 Analyze and evaluate the selected testabtity concepts of the system orequipment design in a qualitative mermer to ensure that the design will support therequired level of testing. Conduct an analysis of the inherent (intrinsic) testebiUty ofthe preliminary design. The anat@s identifies the presence or absence of herdwarefeatures which facilitate testing end identifies problem areas. The method of AppendixB shell be applied to each item identified for inherent te.ratability assessment by therequiring authority. The inputa required by Appendix B shaU be determined by theperforming activity and approved by the requiring authority.202.2.4 Modify the design, es necessary, until the inherent testability equals orexceeds the threshold value. U the performing activity concludes that achieving thethreshold is not possible or coat effective, but that the fault detection end fault isolationrequirements witl be met, the performing activity shatl provide supporting data forreview by the requiring authority.202.2.5 Develop a methodology for Task 203 for predicting fault detection end faultisotetion performance levets.202.3 TASK tNPUT.202.3.1 3ystem or equipment d=ign data.202.3.2 Identification of items to be included in inherent testability analysis(Appendix B).”202.3.3 For each item, inherent testability y threshold value to be achieved. ● Guidancefor establishing a threshotd value is given in Appendix A, paragraph 50.6.11.3.202.4 TASK OUTPUT20%4.1 2ystem or equipment design which incorporates testability features. (202.2.1,202.2.2 and 202.2.4)13

MLL-6TD-2165202.4.2 Description of teatability design tradeoffs and testability features selectedfor implementation; documented in accordance with DI-T-7199. (202.2.2 and 202.2.4)202.4.3 For each item, assignment of weighting factor and scoring method for eachtestability criterion (Appendix B). (202.2.3)202.4.4 Inherent testability y assessment; documented in accordance with DI-T-7199.(202.2.3)202.4.5 Description of methodologies, models and tools to be used in Task 203;documented in accordance with DI-T-7199. (202.2.5)II●*To be specifiedby the requiring authority.14e

●M~2165TASK 203m-~ D~~ DESIGN AND ANALYS12203.1 PURPOSE. To incorporate features into the de-sign of a system or equipmentwhich witl m~tabitity performance requirements and to predict the level of testeffectiveness which witt be achieved for the system or equipment..“203.2 TASK DESCIU~ON203.2.1 incorporate testability design features, including BIT, into the detaUed d=ignfor each Item.‘203.2.2 Ana)yze that atl critical functions of the prime equipment are exercised bytesting to the extent specified. llre performing activity shatt conduct functional testenetysis for each configuration item (Cl) and for each physicet partition of the Cldesignated es a UUT.203.2.3 Conduct en analysis of the test effectiveness of BIT and off-Une test.Identify the failures of each component and the failures betweeneomponentsa~hich correspond to the specified failure modss for each item to be tested.These failures represent the predicted failure population end are the basis for testderivation (BIT and off-line test) and test effectiveness evaluation. Maxtmum use shellbe made of a failure modes and effects analysis (FMEA), from Task 204 of ivllL-STD-470A, if a FMEA is required. ‘he FMEA requirements may have to be modified orsupplemented to provide the level of detail needed.0 b. Model components and interconnections for each item such that thepredicted failure population may be accurately modeled. ‘f%eperforming activity shelldevelop or select modets which are optimum considering accuracy required, cost of testgeneration and simutetion, standardization and commonality.c. Analyze end evaluate the effectiveness of pLsnned testing based uponthe predicted faiIure poputetion. l%e anetysis shetl give particuter emphasis to fauttdetection end fault isolation for critical and high failure rate items end interconnections.‘lIre test effectiveness data shell be used to guide redesign of equipment and testprograms, es required, and to assist in the prediction of spares requirements.d. Prepare justification for any classes of faults which are undetected,cannot be isoteted or are poorly isolated when using the developed test stimuli andsubmit to the requiring authority for review. prepare additional or etternate dkignasticapproaches. Identify hard to test faults to the LSA process.203.2.4 Iterate the design of the prime item built-in test until each predicted testeffectiveness value equals or exceeds the specified vetue.203.2.5 Develop system-level BIT hardware and software, integrating the buUt-in testcapabititias of each subsystem/item.203.2.6 Predict the level of BfT fault detection for the overall system bescd upon theBIT detection predictions, weighted by failure rate, of the individual items, including15

IMIL-BTD-2165GFE. Predict the level of fault isolation for the overall system through system-leveltest. Predict the probability of BIT false alarms for the overall system.203.2.7 Assemble cost data associated with BIT and design for testability on a per ●unit basis (e.g., additional hardware, increased modularity, additional connector pins,etc.). Estract and summarize cost data associated with the implementation of thetestability program, test generation efforts and production test. Provide teateffectiveness predictions es inputs to availability and fife cycle cost analyses.203.2.6 Incorporate BIT and testability corrective design actions es determined bythe maintainability demonstration results and initial testing.203.2.9 incorporate changes and corrections into testability modeb test generation .soft ware, etc., which reflect an improved understanding of operations and failure modeses the design progresses. Use u~ated models, software, etc., to update testeffectiveness predictions as necessary.203.3203.3.1203.3.2203.3.3TASK INPUTIdentification of items to be included in test effectiveness predictions. *System or item preliminary design data. iBIT specification.203.3.4from Task203.3.5Identification of failure modes and effects and failure rates for each item204 of MIL-STD-470A.Test ef festiveness data for GFE. * ●203.3.6203.4Corrective action recommendations from maintainability demonstration.TASK OUTPUT203.4.1 System or item design which meets testability and maintainabilityrequirements. (203.2.1, 203.2.4, 203.2.5 and 203.2.8).’203.4.2as a UnitDescription of built-in test and testabilityUnder Test; documented in appropriatefeatures for each item designatedTest Requirements Document.(203.2.1)203.4.3 Test effectiveness prediction for each item; data provided in support of Task205 of MIL-STD-47 OAand Task 401 of MIL-STD-1388-1A and documented in accordancewith DI-T-7199. (203.2.2, 203.2.3, 203.2.7 and 203.2.9)203.4.4 System test effectiveness prediction; data provided in support of Task 205 ofMIL-STD-470A and documented in accordance with Dt-T-7199. (203.2.6, 203.2.7 and203.2.9)●To be specified by the requiring authority.16I

MIL-51T&2165TASK SECTION S00T’ESl’AND EVALUATION,17

-1IMJL-STD-2165TASK 301TWISTABILITYINPUTS TO MAINTADJAEILITY DEMONS1’EATION301.1 PURPOSE. To determine compliance with specified testability requirementslend essess the velidlty of te.s.tabi~lty predictions.● ’,’301.2 TASK DESCRIPTIONS301.2.1 Determine how testability requirements are to be demonstrated using -maintainability demonstration, test program verification or other demonstrationmethods. The fotlowing elements are to be demonstrated to the extent each is specifiedin the contractual documents:errors.a. The ability of operational system checks to detect the presence ofb. The ability of system or subsystem BIT to detect and isolate failures.equipment.c. The compatibility of each item es a UUT with the selected testisoteted. The ability of the test equipment and associated TPSS to detect andfailures.e. The adequacy of technical documentation with respect to faultdictionaries, probing procedures, manual troubleshooting, theory of operation, etc.f. The correlation of BIT fault detection and fault isolation indications ●with off-tine test results.g. The validity of models used to predict teetabilit y parameters.301.2.2 Develop plans for the demonstration of testability parameters and integrateinto the plans and procedures for maintainability demonstration.301.2.3 Conduct additioml demonstrations, as needed, using the methods and criteriaof MIL-STD-471 and MIL-STD-2077 es appropriate, to obtain sufficient testability datafor evaluation and document as a portion of the testability demonstration results. Thedemonstrations are to be combined with other demonstrations whenever practical.301.3 TASK INPUT301.3.1 Identification of items to be demonstrated. ●301.3.2 Identification of MIL-STD-471 test method or alternative procedure forconducting a maintainability demonstration. *301.3.3 Identification of MIL-STD-2077 quality assurance procedure or alternativeprocedure for evaluation of TPSS.*

MtL-SlW2165● 301.4301.4.1(301.2.2)TASK OUTPUTTestability demonstration plan; documented in accordance with DI-R-7112.301.4.2 Testability demonstration results; documented in accordance with D1-R-7113.(301.2.3)“.●To be specifiedby the requiring authority.19

IMIL-STD-2165APPENDIX ATFSTABILITY PROGRAM APPIJCATfON GULUANCEcof4TRNTs●Paragraph 10.10.1SCOPEPurpose232320.20.1REFERENCED DOCUMENTIssues of documents232330.30.1DEFINITIONSDefinitions242440.40.140.240.340.440.540.640.7GENERAL APPLICATION GUIDANCETaak selection criteriaTestability program inperspectjveSystem testability programItem teatability programCriteria for imposing a testability programduring the D&VphaseEquipment testability programIterations242424253030323250.50.150.1.150.1.250.1.350.1.450.250.2.150.2.250.2.350.2.450.350.3.150.3.1.150.3.1.250.3.250.450.4.150.4.250.4.350.4.4DETAILED APPLICATION GUIDANCETask 101- Testability program planScopeSubmission of planPlan for D&VphaseOrganizational interfacesTestability analysis reportContentUtilizationTRD interfaceClassified dataTask 102- Testability reviewType of reviewProgram reviewsTestability design reviewsAdditional data reviewTask 103- Testability data collection,and analysis planningTestability effectiveness trackingData collection and analysis plansTest maturationOperational test and evaluation3232323234343434343436363636363636363737370;I120●

0MW-SI’D-2165APPENDIX ACONTENTS (continued)PaJ&oParegraph 50.4.4.150.4.4.250.4.4.350.4.550.550.5.150.5.250.5.350.5.450.5.550.5.650.5.750.5.850.5.8.150.5.6.250.650.6.150.6.250.6.350.6.450.6.550.6.650.6.750.6.850.6.950.6.1050.6.1150.6.11.150.6.11.250.6.11.350.750.7.150.7.250.7.350.7.3.150.7.3.250.7.3.350.7.3.4Confirmed failure, BITConfirmed failure, off-lineUnconfirmed failure, BITtestCorrective actionTask 201- Testability requirementsSupportabilityy analysis interfaceNew technologyStandardizationTest objectivesDiagnostic conceptTestability requirementsTestability requirements for systemspecificationTestability requirements for itemspecificationsSystem testUUT testTask 202- Testability preliminary designend anatyaisScope of testabilityD&Vsystem designdesignTest design tradeoffsGeneral. testability issuesUUT end ATE compatibilityBuilt-in testBtT softwereSystem-level built-in testApplication of testability measuresQualitative inherent testabilityInherent testability assessmentevaluationpreliminary design activitiesChecklist scoringThreshold determinationTeak 203- Testability detail designand analysisTestability design techniquesfnherent testability assessmentTest effectiveness measuresFault coverageFault resolutionFault detection timeFault isolation time37363836393939393939404344444444444646474849505152525252545454545454;:5656021

MIL-STO-2165APPSNLMXACONTSNTS (Contiiued)Paragraph 50.7.450.7.550.7.650.7.750.850.8.150.8,250.8.350.8.4Fault modelingSystem level test effectivenessTestability cost end benefit dataIn-service testability measuresTask 301- Testability demonstrationDemonstration parametersBIT and off-line test correlationBIT false alarm rateModel validation565757585858595959Figure 123456ILLUSTRATIONSSystem Testability Program Flow DiagramItem Testabihty Design Detailed Flow DiagramEquipment TestabiUty Program Flow DiagramModel Paragraphs, Preliminary SystemSpecificationModel Requirements, System SpecificationModel Requirements, CI DevelopmentSpecification263133414245Table 1IImTABLRSTeak Application Guidance MatrixTestability Analysis Report ApplicationGuidance MatrixTestability Measure Application GuidanceMatrix29355322

010. SCOPEMLL+71D-2165APPENDIX ATESTABDJTY PROGRAM APPLICATION GUIDANCE-10.1 Purpcse. This appendix provides rationale and guidance for the selection endtailoring of tasks to define a testability program which meets established programobjectives. No contractual requirements are contained in this Appendix.20. REFERENCED 00CUMENTS20.1 Lssues of Documents. The fotlowing documents form a pert of Uris Appendixfor guidance purposes.STANDARDSMILtTARYPUBLICATIONSMIL-STD-781 Reliability Design Quetification end ProductAcceptance Tests Exponential DistributionMIL-STD-1345 (Navy) Test Requirements Documents, Preparation ofMIL-sTo-1519 (uSAF) Test Requirements Documents, preparation ofMIL-STD-2076 UUT Compatibility with ATE, General RequirementforJOINT SERVICENAVMATP 9405DARCOM 34-1AFLCP 800-39AFSCP 800-39NAVMC 272119 March 1981Joint Service Built-in Test Design Guide’Naval Fleet Joint Service Stectronic Design for TestabilityAnalysis CenterTM 824-1628Course NotesI October 1983.TECHNICAL REPORTSNaval Air Systems Avionic Design Guide for ATE CompatibilityCommand1 Auf@t 1979Contract NOO0140-79-C-069623

JAir Force AeronauticalSystems DivisionSeptember 1983Contract F33657-78-C-0502Air Force Rome AirDevelopment CenterDecember 1979RADC-TR-79-309Naval Fleet Analysis Center2 November 1984TM-8243-1685DIRECTIVEDEPARTMENT OF DEFENSE30. DEFIfilTIONSDOD Directive 5000.39MIbSTfl-2165. . . . . . . .Modular Automatic Test Equipment Guide 3-Avionics Testability Design GuideBIT/External Test Figures of Merit and DemonstrationTechniquesTestability Requirements Analysis HandbookAcquisition and Management of IntegratedLogistic Support for Systems and Equipment●30.1 Definitior6. The def initiona included in MIL-STD-1309, MIL-STD-721 andAppendix C sh~ apply.40. GENERAL APPLICATION GUIDANCE ●40.1 Teek selection criteria. The selection of teaks which can materially aid theattainment of testability requirements is a difficult problem for both government andindustry organizations faced with severe funding and schedule constraints. ThisAppendix providea guidance for the selaction of teaks based upon identified programneeds. Once appropriate testability program teaks have been selected, each teak must betailored in terms of timing, comprehensiveness and end products to meet the overallprogram requirements.40.2 TestabiMty prq? eminpspec tive. The planned testability program must bean integral part of the systems engineering process and serve es an important link .between design and logistic support in accordance with DOD Directive 5000.39. Thetasks which influence and are influenced by the testability program are extracted fromDOD Directive 5000.39 in the following paragraphs. .a. Concept exploration phase1. Identify manpower, logistic, reliability, maintainability andteatability parameters critical to system readiness and support costs.2. Estimate what is achievable for each parameter.24●

MU,-STD-2165APPENDIX Aob. Demonstration end validation (D&V)phase1. Conduct tradeoffs among system design characteristics endsupport concepts.2. Establish consistent set of goals and thresholds for readiness,reliability, maintainability, BIT, manpower end logistic parameters.~pport+ekted3. Establish test and evaluation (T&E) plans to SSSeSSachievement ofthresholds.c. Futl state development (FSD) phasemaintainabilityL Perform detailed analysis end tradeoffs of design, reliability end(R&M), manning levets and other logistic requirements.2. Perform T&E of adequacy of planned manpower, supportconcepts, R&M and testability to meet system readiness and utilization objectives.d. Production and deployment phase1. Perform foUow-on evaluation of maintenance ptan, supportcapability, operation and support costs and manpower.a. Correct deficiencies.40.s system testaMfty prourem (PfKILre1). For major systems, the testabilityteaks for each program phesa are summarized in ‘fable I end listed below.Concept eseloration phase - lMablieh testability objectives inprelfminary%ystem specification (Task 201).b. D&VpheSe1. prepare testabitfty program pien (Tesk 101).2. Incorporate testability features into D&V items end evaluateeffectiveness (See 40.4)..3. Select alternativerequirements in system specification.development specifications (Tesk 201).system concept(s). Sstablish testabilityAllocate testability requirements to item102).4. Conduct testabilityc. FSD ~hesereview es pert of system design review (Tesk1, prepare and revise testability y program pian (Tesk 101).25

MIL-STD-2165APPENDIX AT1-—------ ..---,al5.- mU.l----d26

●-“MIL-STD-2165APPENDIX AG‘&N-r—a!s.- mU.------ ---o27

MIL-STD-2165 ,,APPENDIX A~— —-.-=IIr~rl-l-———,——‘-wI I t-LL_Lt—28

MJL-Sl’W2165APPENDIX ATable LTeak Spptication@u“denee matrix.. 101“102103“TaskTestabilityTestabilityprogram ptannirrgreviewsTestabilityy data coUeetionand analysis planningPrcgmxl phaseCON o&v FSD P/D—. — —NA G G NAG1 G G sNA s G G201TestabilityrequirementsG1 G G NA202203Testability preliminarydesign end analysisTestability detail designand enetysisNA s G sNA s G s301TestabUitydemonstrationNA s G sNA - Not applicableG- Generatty eppilcableS - Selectively applicable tohigh r&k iierni during D&V,or to design changes duringP&DCON - Concept ExplorationD&V - Demonstration end ValidationPSI) - Fult Scale DevelopmentP/D - Production/Deplo~ merit1 MIL-STD-1388 k primary implementation document for diegnwtic requirementstradeoffs and review as pert of logistics support anetysis during ConceptExploration phase..29

MIbS1’D-2165APPENDIX Aeffectivenessincorporate testability features into FSD items and evaluate(Se~”40.4).3. Demonstrate system testability effectiveness (Task 301).d. Production and deployment phase - Collect data on achieved testabilityeffectiveness. Take corrective action, es is necessary.40.4 Item testability progtam (Figur e 2). For afl items, whether developed as asubsystem under a system acquisition program or developed under an equipmentacquisition program, the testab~lt y tasks are listed below.a.Preliminarydesignpert of a system202).1. Prepare testability program plan, if a plan was not developed asacquisition program (Task 101).2. Incorporate testability features into preliminary design (Tesk(Task 102).b.predict inherent(Task 102).3. Prepare inherent testability checklist for each item (Tssk 202).4. Conduct testability review as part of preliminary design reviewDetail designIncorporate testability features into detail design (Tesk 203), and~~stability for each item (Task 202).2. Predict test effectiveness for each item (Task 203).3. Conduct testability review as part of the critical design review4. Demonstrate item testability effectiveness (Tesk 301).● ✛40.5 Criteria for imps” mg a teetabtit y program during the D&Vphase. During the -D&V phase, a formaf testability program should be apphed to the system integrationeffort and, in addition, should be selectively applied to those subsystems which present ahigh risk in testing. The high risk aspect of test design may be a result of:a. Criticality of function to be tested,b. Difficulty of achieving desired test quality at an affordable cost,c. Difficulty of defining appropriate testability measures or demonstrationsfor technology being tested,30

●MIL-STD-2165APPENDIX A—-IQ&— —31

MIL-STD-2165APPENDIX Ad. Large impact on maintainability or elements if expected test qudlty,automation, throughput, etc., is not achieved, or●limited.e. High probability that modifications to the subsystem during FSD will be40.6 Equipment testability progr am (Pigur e 3). For the acquisition of less-thanmajorsystems or individual equipments, the testability tm.ks are listed below.a. Establish system or equipment testability requirements (performed byrequiring authority using Teak 201 as guidance).b. Prepare testability program plan (Task 101).(See 40.4).c. Incorporate testability features into items and evaluate effectivenessd.” Collect data on achieved t=tability effectivenss.s (performed byrequiring authority using Task 103 as guidance).40.7 Iteretiors. Certain tasks contained in this standard are highly iterative innature and recur at various times during the acquisition cycle, proceeding to lower levelsof hardware indenture and greater detail in tha classical systems engineering manner.50. DETAILED APPLICATION GUIDANCE50.1 Task 101- Testabtity prcgram @anni~50”1”1 =“ The testability program plan is the basic tool for establishing andexecuting an effective testability program. The testability program plan shoulddocument what testability tasks are to be accomplished, how each teak will beaccomplished, when they will be accomplished, and how the results of the task will beused. The testability y program plan may be a stand-alone document but preferably shouldbe included as part of the systems engineering planning when required. Plans assist therequiring authority in evaluating the prospective performing activities approach to andunderstanding of the testability task requirements, and the organizational structure forperforming testability tasks. The testability program plan should be closely coordinatedwith the maintainabiliaty program plan.●50.1.2 Submission of plan. When requiring a testability program plan, the requiringauthority should allow the performing activity to propose specifically tailored tasks withsupport ing rationale to show overall program benefits. The t es.tabilit y program planshould be a dynamic document that reflects current program status and planned actions.Accordingly, procedures must be established for updates and approval of updates by therequiring authority when conditions warrant. Program schedule changes, test results, ortestability task results may dictate a change in the testability program plan in order forit to be used effectively es a management document.3’2●

L —. — -1I-——— ——__ i33

MIL-STD-2165APPENDIX A50.1.3 Plan for D&V phase. When submitted at the beginning of a D&V phase, thetestability program plan shoufd highlight the methodology to be used in establishingqualitative and quantitative testability requirements for the system specification. Theplan should also describe the methodology to be used in allocating quantitative systemtestability requirements down to the subsystem or configuration item level. The natureof the D&V phase will vary cormiderably from program to program, ranging from a,Ifirming up,! of preliminary requirements to a multi-contractor “fly Off” of cOmpetingalternatives. In alf cases, sufficient data must be furnished to the Government to permita meaningful evaluation of testing and testability alternatives. The testability pregramplan should indicate how the flow of information is to be accomplished: through informalcustomer reviews, through CDRL.data submiesioms, and through testability reviews es anintegral part of SDR.● ’150.1.4 Organizational interfaces. In order to establish and maintain an effectivetestability program, the testability manager must form a close liaison with all designdisciplines. In satisfying system support requirements, the prime system design must betreated as one of the elements which may be traded off through the supportability.analysis process. As a result, the testability manager must be prepared to workaggressively with design engineers to ensure a proper balance between performance, costand supportability. It is not efficient or effective for the testability manager to assumethe role of post-design critic and risk large cost and schedule impacts. The testabilityinfluence must be apparent from the initiation of the design effort, through designguidelines, training programs, objective measures, etc.50.2 Testability arialyeis report50.2.1 Content. The testability analysis report collects a number of differenttestability ~ults and documents the results in a single, standard format. Thetestability analysis report should be updated at least prior to every major programreview and this requirement must be reflected in the CDRL. The actual content andlevel of detail in each submission of the testability analysis report wifl depend upon theprogram phase. Table If provides guidance es to which tasks/subtasks would be includedfor review by the requiring authority prior to four specific reviews. The first entry for asubtask indicates the time of initial submission. Any following entries indicate one ormore updates to the originef data es the design becomes defined in greater detaiL50.2.2 Utilization. The testability analysis report should be used by the performingactivity to disseminate all aspects of the testability design status to the variousorganizational elements. As such, the testability analysis report should be considered tobe a dynamic document, containing the latest available design information and issuedunder an appropriate degree of configuration control. As a minimum, the testabilityanalysis report should accurately reflect the latest design data when informal t esta.bili tydesign reviews are held.50.2.3 TRD interface. The testability analysis performed during the FSD phase anddocumented in the testability y analysis report should be used as a partiaf basis for theTRD for each UUT. The TRD, developed in accordance with MIL-STD-1519 or MIL-STD-1345, constitutes the formal interface between the activity responsible for detailed34● ’

MU-SI’D-2165APPENDIX ATable IL TestabilityAnatyata Report eRPtieatton RUI “dance matrixSubtask D@mt SDR PDR CDB— x201.4.2 Requirements tradeoffs x x202.4.2 Design tradeoffs x x x202.4.2 Testability design data x x202.4.3 Inherent testability checklist x202.4.4 Inherent testability assessment x x202.4.5 Detail design analysis procedures x x203.4.3 Item test effectiveness prediction x x203.4.4 System test effectiveness x xpredictionNote: Each submission of the Testability Analysis Report should be required by theCDRL to be delivered sufficiently in advance of each review such that the requiringauthority may review the materiat.35

MIL-SfD-2165APPENDIX Ahard ware design and the activity responsible for TPS development. This document servesas a single source of aff performance verification .wd diagnostic procedures, and for all ●equipmentsupportedrequirements to support each UUT in its maintenancemanually or by ATE. The TRD also providesenvironment, whetherdetailed configurationidentification for UUT design and test requirements data to ensure compatible testprograms.50.2.4 Classified data. If classified data is required to document the testability .analysis, it should be placed in a separate attachment to the testability analysis reportsuch that the testability analysis report may have the widest possible distribution.50.3 Teak 102- Testability review50.3.1 ,~{ This task IS “ directed toward two types of review: (1) formalsystem program reviews Subtask 102.2.1), and (2) review of design information withinthe performing activity from a testability standpoint (Subtask 102.2.2). The second typeprovides testability specialists the authority with which to manage design tradeoffs. Formost developers this type of review is a normal operating practice. Procedures for thistype of review would be included in the testability program plan.50.3.1.1 Program reviews. System program reviews such as preliminary designeviews are an important management and technical tool ofthe requiring authority. They should be specified in statements of work to ensureadequate staffing and funding and are typically held periodically during an acquisitionprogram to evaluate overall program progress, consistency, and technical adequacy. Anoverall testability program status should be an integral part of these reviews whetherconducted with subcontractors or with the requiring authority.●50.3.1.2 TestabUty design reviews. Testability design reviews are necessary to assessthe progress of the testability design in greater technical detail and at a greaterfrequency than is provided by system program reviews. The reviews shell ensure that thevarious organizetionaf elements within the performing activity which impact or areimpacted by testability are represented and have an appropriate degree of authority inmaking decisions. ‘l%eresults of performing activity% internal and subcontractor systemreviews should be documented and made available to the requiring authority on request.These reviews should be coordinated, whenever possible, with maintainability, ILSMT andprogram management reviews.50.3.2 Additional data review. In addition to formal reviews, useful information canoften be gamed from performmg activity data which is not submitted formally, butwhich can be made available through an accession list. A data item for this list must beincluded in the CDRL. This list is a compilation of documents and data which the “requiring authority can order, or which can be reviewed at the performing activity’sfacility.50.4 Task ]03 - Testability datJ3 COllSCtiOll $llld sldysfs @9WIli~50.4.1 Testability effectiveness tracking. A testability program cannot be totallyeffective unless provisions are made for the systematic tracking and evaluation oftestability effectiveness beyond the system development phase. The objective of this36

o tasktdIbS1’D-2165APPENDIX Ais to plan for the eveiuetion of the impact of actuai operational end maintenanceenvironments on the abiiity of production equipment to be tested. The effectiveness oftestability design techniques for intermediate or depot level maintenance tasks ismonitored end anaiyzed es part of US”Mevacuation. Much of the actual collection andenelvsis of date and resuitine corrective actions mav occur bevond the end of thecon~act under which the tes”hbitity program is im~ed end ma; be accomplished bypersonnel other than those of the performing activity. Stiii, it is essentiet that thepfanning for this task be initiated in the FSD phase, preferably by the critical designreview.50.4.2 Data coUection and anatysis ptans. separate ptens should be prepared fortedabilitv data collection and enelvsis durhw (t) mwduction DhSSe(Subtesk 103.2.1) and(2) deplo~ment phase (Subtesk 103.~.2). The ~~ns-should cle~ly delineate which enaiysisdata are to be reported in various documents, such es T&E reports, production testreports, factory acceptance test reports, etc..●.50.4.3 Test maturation. Most test implementations, no matter how wetl conceived,require a period of time for identification of problems and corrective action to reachspecified per for mence leveis. Thii ‘maturing” process eppiies equally to BIT end off-linetest. ‘11-Iisis especially true in setting test tolerances for BIT end of f-ibse test used totest analog parameters. The setting of test tolerances to achieve an optimum batencebetween failure detection and fatse aierms usuaUy requires the logging of considerabletest time. It should be emphasized, however, that the necessity for “fine-tuning” a testsystem during production and deployment in no way diminishes the requirement toprovide a %est possdbie” design during FSD. One way of accelerating the testmaturation process is to utilize ptemed field or depot testers for portions of acceptanceteat. BIT test hardware and softwere should be exercised for those failures discoveredand the BIT effectiveness documented and assessed.50.4.4 Operational test and evaluation. The suitability of BiT should be assessed esen integral pert of operattonel test aad evaluation. A closed-loop date trackhtg systemshould be implemented to track initief failure occurrences, organizational-levelcorrective actions, subsequent higher-ievel maintenance actions, and subsequentutilization end performance of repaired and returned items. The data collection must beintegrated es much as possible with simiter data collection requirements such es thosefor tracking reflabitity end maintainability. The data trackimr svstem must collectsufficient data to eup~rt the analysis of %0.4.4.1 through 50.4~4.{. AN maintenanceactions are first reviewed to determine if the failed item is relevant to BIT or off-linetest. For example, items with loose bolts are not relevant to testability aneiysis. if atsome point in the data tracking, en actual failure is found, the analysis for confirmedfailures (50.4.4.1 and 50.4.4.2) is applied. If en actuai faiiure is not found, the enetysiefor non-confirmed failures (50.4.4.3) is eppiied.50.4.4.1 Confirmed failure, BiT. For each confirmed failure, data on BITeffectiveness are analyzed:a. Did BIT detect the failure?b. Did BIT correctly indicate operational status to the operator?37

1MfL.+TW2165APPENDIX Ac. Did BIT provide effective fault isolation information for corrective ●maintenance actions?d. What was the ambiguity size (number of modules to be removed orfurther tested) due to fault localization or isolation by BIT?e. How much time was required for fault isolation at the organizationallevel of maintenance?50.4.4.2 Confirmed failure, off-line test. For each confirm ed failure, data on off-linetest compatibility are analyzed:a. Were any workarounds required to overcome mechanical or electricaldeficiencies in the UUT and ATE interface?b. Did the ATE system provide failure detection results consistent withthose of the initiaf detection by BIT?c. Did the UUT design inhibit the ATE system from providing accuratefault isolation data?50.4.4.3 Unconfirmed failure, BIT. For each unconfirmed failure situation (cannotduplicate) resulting from a BIT indication or alarm, the following data are analyzed:a.b.c.d.What is the nature of the alarm?What is the frequency of occurrence of the alarm?What failure or failures are expectedto cause the observed alarm?What are the potential consequences of ignoring the alarm (in terms ofcrew safety, launching unreliable weapons, etc.)?●Ie.What are the operational coats of responding to the false alarm (interms of aborted mission% degraded mcde operation, systemdowntime)?What are the maintenance costs associated with the false alarm?What add~tionel data are available from operational software dumps(e.g., soft failure occurrences) to characterize cannot duplicate . ~occurrences?50.4.5 Corrective action. The data on BIT effectiveness and off-line testcompatibility are summarized and corrective action, if needed, is proposed by theperforming activity or user activity. Those corrective actions dealing with redesign ofthe prime system aresubmitted forreview and implementation as pert of the establishedengineering change process.438

MU#t’D-2165APPENDtX A● 50.5 Teats 201- Testability requs“rementa50.5.1 supportabiti ty analysis interface. It is essential to conduct a Lqfiitic SupportAnsdvsis or other sucioortabilitv enetvses (Subteek 201.2.1) early in en acquisitionpr~”arn to identify “c~htrainti, ‘thresh~lds, and targets for improvement, end to providesupportability input into early system tradeoffs. It Is during the early phases of anacquisition program that the greatest opportunity exists to hsfluance the system design- from e supportability standpoint. These analyses can identify supportability andmaintainability parameters for the new system which are reasonably attainable, elonswith the prim: ~ivers of suc.port, manpower, personnel end training, cost, and readiness.The drivers, once identified,- provide 8 bes~ for concentrated analysis effort to identifytargets and methods of improvement. Mission and support systems definition tasks aregeneratly conducted at system end subsystem levets early in the system acquisitionprocess (Concept and D&V phases). Identification and analysis of risks plays a key roledue to the high level of uncertainty end unknowns early in a system% tife cycle.Performance of these tasks requires examination of current operational systems endtheir characteristics es wetl es projected systems and capabitit ies that will be aveitablein the time frame that the new system wilt reach its operationef environment. Newsystem supportability and supportability reteted design constraints must be establishedbaaed upon support systems and resources that will be avaitable when the new system isfielded. Additionet guidance may be found in the Testability Requirements AnalysisHandbook (20.1).●50.5.2 New technology. Subtesk 201.2.la identifies new system supportabilityenhancements over existing systems through the application of new technology. Thisanalysis will specify where improvements can be reatized. It wilt identify the expectedeffect of improvements on supportability, cost and readiness vetues so that testequipment end testabiti ty objectives can be eetabtished. This task should be performedby design personnel in conjunction with supportability specialists.50.5.3 Standardization. In many cases, utilization of existing support resources endpersonnel sisitts (Subtesk 201.2.lb) can substantially reduce a system% tife cycle cost,enhance the system% readiness, minimize the impact of introduction of the new system,and increase the mobility of the operational unit using the new system. Te@ systemstandardization requirements for new systems can also artse from Department ofDefense or service suptmrt policies. Examples of these requirements can includestandard software language requirements or use of standard multi~ystem testequipment.The type of parameter (e.g., objectives, goats, thresholds)’ de%~d ~~ 0 per forming Task 201 will depend on the phase of development.Generatly, during the Concept Exploration phase testing objectives witt be ast.sbtished(Subtask 201.2.2). These objectives are established baaed on the results of previousmission and support systems definition tasks, especietty the opportunities identified es aresult of new technology, and are subject to tradeoffs to achieve the most coat effectivesolution to the mission-need.50.5.5 Diegnostic concept. The development of the diegrmtic concept begins in theConcept Exploration phase and is refined es more detailed data becomes avaiteble duringthe D&V phase. Quantitative testability requirements would not normally be specifiedo39

1MIL+l!D-2165APPENDfX Aduring the concept exploration phase. However, a preliminary estimate of the critical ●testability parameters should be made to ascertain if the required system availabilityand maintenance and logistic support concepts can be supported using testabilityparameters demonstrated es achievable on similar systems. The diagnostic conceptusually evolves as follows:Determine on-line BIT requirements to ensure monitoring of criticalfunctions ar% monitoring of functions which affect personnel safety. .availabilityb. Determine additional on-line BIT requirements to support high systemthrough redundant equipments and functions, backup or degraded modes of .operaflon, etc.c. Determine additional BIT requirements to support confidence checksprior to system initiation or at periodic intervals during system operation.As more detailed design data becomes avaifable, usually during the D&V phase, thediagnostic concept further evolves, making extensive use of readiness and life cycle costmodels:d. Determine what fault isolation capability is inherent in (a) through (c);determine additional BIT requirements to support the preliminary maintenance concept.Determine automatic, semi-automatic end manual off-line testrequiremen~ to fill voids due to technical or cost limitations associated with using BIT.Note: The sum of the requirements for BIT, off-line automatic test, semi-automatic test ●and manual test must always provide for a complete (100%) maintenance capability ateach maintenance level.f. Determine what existing organizational level BIT capabilities are usableat the next higher maintenance level. Determine requirements for ATE. @enerelpurpose electro~ic test equipment (GPETE) end technical ~ocumentation to provid~~””withBIT, the total maintenance capability.If testability requirements are included in a preliminary system specification, theyshould be qualitative in nature pending the more quantitative tradeoffs of the D&Vdesign. Figure 4 provides some model paragraphs which may be included primarily toalert the perform ing activity that design for testability is considered to be en importantaspect of design end that quantitative requirements will be imposed in the final systemspecification. Alternatively, the model paragraphs for the system specification, figure5, could be used in the preliminary system specification with all quantitativerequirements “to be determined.”S0.5.6 Testability requirements. Prior to the Full Scale Development phase, firmtestability requirements are established (Subtasks 201.2.5) which are not subject totradeoff. These represent the mini mum - essential levels of performance that must besatisfied. Overall system objectives, goals and thresholda must be allocated andtranslated to arrive at testability requirements to be included in the system specification40i

MtkfH’D-2165APPRNDIX A3.x.xDesign for testability3.X.X.1 Pertitioni~. The system shall be partitioned based, in pert,upan the ability to confidently iaotate fautta.Test ointe. Each item within the system shall have sufficient3“X”X”2 test points + or the measurement or stimutus of intemet circuit nodes so asto achieve en inherently high level of fault detection and isotation.3.x.x.3 Maintenance capability. For each level of maintenance, BIT,off-tine automat ic test end menuat test c8pabiU ties shell be integrated toprovide a consistent end complete maintenance capability. The degree oftest automation shell be consistent with the proposed personnel skittlevets end corrective end preventive maintenance requirements.3.x.x.4 ~. Mission critical functions shall be monitored by BIT. BfTtolerances shetl be set to optimize fault detection end false alarmcharacteristics. BtT indicators shatt be designed for maximum utilizationby intended personnel (operator or maintainer).FQure 4. Modet perewwhs, pre timinery system specification.41

LIMIW%PD-2165APPENDIX A3.x.xDesign for testability ya. Requirement for status monitoring.b. Definition of failure modes, including interconnectionfailures, specified to be the basis for test design.c. Requirement for failure coverage (% detection) usingfull test resources.d. Requirement for failure coverage using BIT.Requirement for failure coverage using only themonitoring ~f operational signals by BIT.f. Requirement for maximum failure latency for BIT.% Requirement for maximum acceptable BIT false alarmrate; definition of false alarm.using BIT.fL Requirement for fault isolation to a replaceable itemi. Requirement for fault isolation times.●Ij. Restrictions on BIT resources in items of herdwere size,weight and power, memory size end test time.k. Requirement for BIT hardwere reliability.L Requirement for automatic error recovery.m. Requirement for fault detection consistency betweenhardwere levels and maintenance levels.Figure 5. Model reqru“remcnte, eyetern apecification.42●(

MfIATTD-2165APPENDIX Aor other document for contract compliance (Subtesk 201.2.6). This subtask is necessaryto assure that system specification or contract parameters include onty those parameterswhich the performing activity can control through design end support systemdevelopment. The support burden and other effects of the government furnishedmaterial, administrative and logistic detey times, and other items outaide the control ofthe performing activity must be accounted for in this process.50.5.7 Testebllity requirements for system specification. Quantitative testabihtyrequire mentZZ analys during the D&V phase and areincorporated in the system specification. Requirements may be expressed in terms ofgoals andthresholrb rather than esa single number. Model requirements for tsstebtityinasystem specification are provided in Figure 5andere discussed below.The system specification includes testability requirements for failuredetection, failure isolation and BIT constraints. Requirement (a) defines the interfacebetween theprimesyetem and an external monitoring eystem, if applicable. Perticutarattention should be Riven to the use of BIT circuitry to provide performance and statusmonitoring. Requir~ment (b) provides the basis ~or alt subsequent t-t design endevaluation. Failure modes are characterized besed upon theeomponent technology used,the assembly process used, the detrimental environment effects anticipated in theintended application, etc. Maximum use should be made of prior retiabitity endysiamdfault analysis data such as FMEA end fault trees. The data represent a profile ofestimated system failures to beconstantly refined andupdetedm the design progresses.Requirements (c) through (e) dest with test approaches. Requirement (c)permits the use of all test resources end, es such, should etways demand 100% failurecoverage. Requirement (d) indicates the proportion of failures to be detectedautomaticaUy. Excluded failures form the basis for manual troubleshooting procedures(swapping large items, manual probing, etc.). Requirement (e) is a requirement forcleating quickly with critical failures and is a subset of (d). The failure detectionappr~ch selected is based upon the requirement for maximum acceptable failurelatency. Concurrent (continuous) failure detection techniques (utilizing herdwereredundancy, such es parity) are specified for monitoring those functions which aremission critical or affect safety and where protection must be provided against thepropegationof errors through the system. ‘t%e manmum permitted failure tetency forconcurrent failure detection end other cfesses of automatic testing is imposed byrequirement (f). This requirement determines the frequency at which periodic diagnosticsoftwere, etc. will run. The frequency of periodic end on-demand testing is based uponfunction, failure rates, wear out factors, maximum acceptable failure tatency, end thespecified operational and maintenance concept.Requirement (g) is the maximum BIT feIse eterm rate. A~rms which OCC~during system operation but cannot be later duplicated may actualfy be intermittentfailures or may indeed be a true probtem with the BtT circuitry. It may be useful to usethe system specification to require sufficient instrumentetion in the system to allow thesorting out end correction of real BIT problems (e.g., BIT faults, wrong thresholds, etc.)during operational test and evaluation.Requirement (h) requires fault isotation by BIT to a subsystem or to a lowerIlevel part, depending upon the maintenance concept. This requirement is usuatly43

MfL-SID-2165APPENDIX Aexpressed es “fault isolation to one item X% of the time, fault isolation to N or fewer ●items Y% of the time~’ Here, the total failure population (100%) consists of thosefailures detected by BIT (requirement (d)). The percentages should always be weighted 4by failure rates to accurately reflect the effectiveness of BIT in the field.Requirement (i), fault isolation time, is derived from maintainabilityrequirements, primarily Mean Time to Repair (MTTR) or repair time at some percentile.Fault isolation time = repair time - (preparation time + disassembly time +interchange time + reassembly time + alignment time + verificationtime)The preparation time, disassembly time, interchange time, reassembly timeend alignment time may be estimated. The verification time is usually equal to thefailure detection test time in that the same tests are likely to be used for each.Requirement (j), BIT constraints, should not be arbitrarily imposed but shouldbe consistent with the BIT performance specified in requirements (a) through (i).Historically, systems have needed atmut 5 to 20% additional hardware forimplementations of adequate BfT. However, for some systems, 1% may be sufficientwhereas other systems may need more than 20%.Requirement (k), BIT reliability, again should not be arbitrarily imposed butshould be consistent with the required BIT performance. This requirement may also beused to specify those critical functions for which a failed BIT must not interfere.50.5.6 Testability requirements for item specifications. Testability requirements ●for configuration items (CLS)support two dktinct requirements: system test (primarilyBIT) end shop test (ATE and GPETE). Model requirements for testability are presentedin Figure 6.50.5.8.1 System test, Quantitative testability requirements for each CI are allocatedfrom system teat ability requirements baaed upon refative failure rates of Cfs, miaAoncrit icafi ty of CJS or other specified criteria. In many digital systems, BIT isimplemented, in whole or in part, through software. Here testability requirements willaPPe~ in a comPuter Program configuration item (CpCI) development specification. Theprogram may be dedicated to the BIT function (i.e., a maintenance program) or may be amission program which contains test functions.50.5.8.2 UUT test. Shop test requirements are determined by how the CI is furtherpartitioned, If at all, into UUTS. Testability requirements for each UUT should be .included in the appropriate CI development specification.50.6 Teak 202- Testability prelim”mary design and analysis50.6.1 Scope of testabifi ty design. Testabilityy addresses three major design areas:a. The compatibility between the item and its off-line test equipment44●1I!II

’MIL+I’D-2165APPENDIX AThe fotlowing Cl requirements are allocated from system requirements:a. Definition of fault modes to be used es the basis for test design.b. Requirement for fault coverage using fufl test resources (Note:Requirement should be 100% fault coverage).Requirement for fault coverage end failure reporting using fullBIT r~urces of CLd. Requirement for fault coverage and failure reporting using onlyBIT monitoring of operational signets within CI.Requirement for maximum failure tetencymonit~ring of criticet signals.for BIT and operationalf. Requirementfor reporting e transientfor maximum BIT false alarm rate. include criteriaor intermittent fault es a valid BIT atarm.g. Requirement for fault isolation to one or more numbers of subitemsusing Cl BIT.h. Requirement for CI fault isotetion times using BfT.Restrictions on BIT resources.; Requirement for BIT hardwere reliability.Requirement for periodic verificationsensors.of calibration of BITThe foUowing requirements apply to each partition within theconfiguration item which is identified es a UU~Requirements for compatibility (functional, electrical, mechenicel)b~ween the UUT and the selected ATE.b. Requirements for test point access for the UUT.c. Requirements for fault coverage using fufl test resources of theintermediate or depot maintenance facilities. (Note: Requirement should be100% fault coverage).(ATE ~dRequirement for faultTPS Plus embedded BIT).coverage using automatic test resourcestests ;ingRequirement for averageautomatic teat resources.(or maximum) test time for GO/NO GOf.resultingRequirement for maximum ratein cannot duplicates and retestof false NO GO indicationsokeys using automatic testresources.g. Requirement for fault isolation to one or more number ofsubitems within the UUT using automatic test resources.h.resources.Requirement for fault isolation times using eutomatic testNOTIk A UUT at en intermediate maintenance facility may containseveral smelfer UU~ to be tested at a depot maintenance fecilityand all should be Usted in the Cl specification.Figure 6. Modef reqtu“rements, Cl developmenteisecifieation.45

IIMflrST’D-2165APPENDIX Ab. The BIT (herdware and softwere)isolate faultsprovided in the item to detect andaThe structure of the item in terms of (1) partitioning to enhance faultisolation an~(2) providhrg access and control to the tester (whether BIT or off-line test)for internal nodes within the item toenhance fault detection end isolation.Testability concepts, when applied to weapon system and support system .designs, have thepotentiel to:a. Facilitate the development of high quality testsb. Facilitate manufacturing testc. Provide a performance monitoring capability (through BIT)manualsd. Facilitate the development of fault isolation procedures for technicale. Improve the quality end reduce the cost of maintenance testing andrepair at all levefs of maintenance.Subtask 202.2.1 requires the performing activity to integrate testability into the designprocess. Severaldesign techniquesdesign guides and handbooka are available which explain testabilitywhich have proven successful in certain applications. (See Section 20,publications). The following paragraphs provide a summary of some testability designissues.50.6.2 D&V system deaiSSS. During the D&V phese,alternate system designs areevsfuated. This includes the analysis of manpower requirements, support costs,reliability, maintainability, and system readiness. There are usually no detailed,quantitative specifications for testability in D&Vsystem designs. fn fact, the purpose ofthe D&V phase, with respect to testability, is to determine quantitative testability yrequirements that are achievable, affordable, and adequately support system operationand maintenance. In making this determination, it is reasonable to apply Task 202 toselected items to be implemented in the alternative systems. These items may beselected because they have potential testing problems or are not expected to be modifiedduring FSD.●50.6.3 Test design tradeoffs. The overall test design will usually incorporate a mixof BIT, off-tine automatic test and manual test which provides a level of test capabilityconsistent with operational availability requirements and fife-cycle cost requirements.Alternate designs are analyzed and traded off against requirements of performance,supportability, and cost to arrive at a configuration best meeting the requirements atminimum cost.IIIa. Manual or automatic test tradeoffs. Decisions regarding the type oftest equipment to be used for system morutoring anf maintenance are made based u~nrepair policies, overall maintenance plans and planned number of systems. Tradeoffs are46●I

I●MUASt’W2155APPENDIX Amade for test requirements at each maintenance level, considering test complexity, timeto fault isolate, operationet environment, logistic support requirements, developmenttime and cost. ‘the degree of testing automation must be consistent with the ptennedskill Ievets of the equipment operators and maintenance personnel.I.b. BIT or ATE tradeoffs. Within the category of automatic testing, theallocation of requirements to BIT or off-tine test is driven by the natural differencesbetween BIT capebitities and ATE capabilities:1. BIT is used to provide initial fault detection for a system orequipment and to provide initial fault isotation to a smetl group ofitems. BIT has the advantage of operating in the missionenvironment and being self%uff icient.2, Off-tine ATE is used to provide fault detection for an item es aUUT and provide fault isotetion to components within an item.ATE does not impose the weight, volume, power and reliabilitypenalties on the prime system that BIT does.Ic. Coordination. in developing off-tine test desigrw, maximum utilizationshould be made of available BtT capability within each UUT. In addition, test tolerancesused by off-tine test should be tighter than those used by BtT to avoid the ‘retest Okeynproblem.50.6.4 General testability issues. Testability features in a system or item designsupport both BIT and off-tine test.a. Physical partitioni~. The ease or difficulty of fault isolation dependsto a large extent upon the size and complexity of replaceable items:1. The physicat partitioning of a system into items should be based,in pert, upon the enhancement of the fault isotetion process.2. The maximum number of item pins must be consistent with theinterface capabilities of the proposed ATE.1I3. Items should be limited to onty analog or onty digital circuitry,whenever practicai, and when functional partitioning is notimpaired.4. Where practicet, circuits belongimg to an inherently largeambiguity group due to signet fan-out should be ptaced in thesame package.b. Punctionel partitioning. Whenever possible, each function should beimplemented on a singie repteceeble item to maka fault isotetion straightforward. Ifmore then one function is placed on a replaceable item, provisions shoutd be made toallow for the independent testing of each function.:047

MfL-STD-2165APPENDfX Ac. Electrical partitioni Whenever possible, the block of circuitrycurrently being tested should from circuitry not being tested through the useof blocking gates, tri-state devices, relays, etc. This “divide and conquer” approach is●baaed upon the concept that test time increases exponentially with the complexity of thecircuit.d. Initialization. The system or equipment should be designed such that ithas a well-defined initief state to commence the fault isolation process. Non- -achievement of the correct initiaf state should be reported to the operator along withsufficient signature data for fault isolation. The system or equipment should be designedto initialize to a unique state such that it will respond in a consistent manner for .muItipIe testing of a given failure.Module interface. Maximum use should be made of available conneetorpins to inc~rporate test control and access. For high density circuits and boards,preference may be given to additionef circuitry (e.g., multiplexer, shift registers) ratherthen additional pins.f. Test control (controllability). Special test input signata, data paths, endcircuitry should be incorporated to provide the test system, whether BIT or ATE,sufficient control over internal item or component operation for the detection andisolation of internef faults. Special attention is ~ven to the independent control of clocklines, clear lines, breaking of feedback loops, end tri-state isolation of components.I& Test access (observabi~ty). Test points, data paths, and circuitry●should be incorporated to provide the test sys tern, whether BIT or ATE, sufficientsignature data for fault detection and isolation within the item. The selection of]Iphysical (real) test points should be sufficient to accurately determine the value ofinternal nodes (virtual test points) of interest. There should be no requirement to probeinternal points for organizational-level fault isolation.h. Parts selection. In selecting between parts, each. with satisfactory(performance characteristics, preference is given to integrated circuit components andassembled items which have satisfactory testability characteristics. Preference is given Ito those integrated circuits for which sufficient disclosure of internet structure andfailure modes has been provided es a basis for effective, economical testing. Ii. Failure mode characterization. Test design end testability design -features which support testing should be based upon expected failure modes and effects.Failure modes (e.g., stuck faults, open faults, out-of-tolerance faults) must be Icharacterized based upon the component technology used, the assembly process used andthe detrimental environment effects anticipated in the intended application. Maximumuse is made of prior analysis and government data. IIS0.6.5 Each UUT should be designed such that it iswith selected or available ATE so es to reduceor eliminate the need for a large number of unique interface device designs. ff the ATEhas not been selected, the general compatibility requirements of MIL-STD-2076 may beaeptied to the UUT and ATE design as appropriate.48●I\

IIIMIfnTTD-2165APPENDIX Alo a. Electricet pertitioni ng for off-line test. The ATE should have sufficientcontrol over the electrlcat partitioning or UT such that relatwely smell,independent, and manageable blocks of circuitryemay be defined es the basis of testI derivation, test documentation, and teat evaluation. The UUT design should supportruining individual test program segments on en ATE independent of other test programsegments.1- b. UUT test point selection. The number end placement of UUT testpoints is based upon the following:. LTest points are selectedbased upon fault isolation requirements.o2.3.4.5.6.7.8.9.Test points selected are readily 6ccessible for connection to ATEvia system/equipment connectors or test connectors.Test points are chosen so that high voltage end currentmeasurements are consistent with safety requirements.Test point meewremente relate to a common equipment ground.Test points are decoupled from the ATE to assure thatdegradation of equipment performance does not occur es a resultof connections to the ATE.Test ~ints of high voltage or current are physically isolated fromtest points of low logic level signals.Teat points are selected with due consideration for ATEimplementation and consistent with reasonable ATE frequencyrequirements.Test points are chosen to segregate analog end digital circuitryfor independent testing.Test points are selected with due consideration for ATEimplementation end consistent with reasonable ATE measurementaccuracies.SO.6.6 Built-in test. 3uitable BfT features are incorporated into each item toprovide init~el fault detection and fault isotetion to a subitem. This BIT may also beutilized to verify the operability of the system following a corrective maintenanceaction or a reconfiguration into a degraded mode.a. BtT fault detection approaches. Fault detection approaches may becategorized es concurrent (continuous) , monitoring, periodic, and on-demand techniques.The selection is based upon the requirement for maximum acceptable fault tetency.Concurrent (continuous) fault detection techniques (utitizing herdware redundancy) areused for monitoring those functions which are mission critical or affect safety end where49

IIMIL-BTD-Z165APPEF3DLXAprotection must be provided against the propagation of errors through the system. ●Periodic testing is used for monitoring those functions which provide backup or standbycapabilities or are not mission critical, On-demand testing is used for monitoring thosefunctions which require operator interaction, sensor simulation, and so forth, or whichare not easily, safely, or cost-effectively initiated automatically. The frequency endlength of periodic end on-demand testing is based upon function, failure rates, wear outfactors, meximum acceptable failure latency, andthespecified maintenance concept.b. Electrical partitioning for BIT. The BITcircuitry should have sufficientcontrol over the electrical partitionimz of the item such that relatively small,independent, and manageable “blocks of ‘circuitry can be defined as the basis of testderivation, test documentation, and test evaluation. Ln particular, for computer-basedequipment, such control should be made available to BIT software.below:c. BfT design tradeoffs. Some of the BIT design tradeoff ieeues are listed1. Centralized versus distributed BIT.2. Tailored versus flexible BIT.3. Active stimulus versus passive monitoring.4. Circuitry to test BIT circuitry.5. Hardwere versus software versus firmware BIT.6. Placement of BIT failure indicators.I50.6.7 BIT software. BIT software includes confidence tests (GO/NO GO tests) forfault detection and diagnostic tests for fault isolation. 1● ,a. BIT memory sizing. When estimating memory requirements for a {digital system, it is essential that sufficient words are reserved: I1. fn control memory for the storage of micro-diagnostics andinitialization routines. I2. fn main memory for the storage of error processing routines and 1confidence tests.3. In secondary memory (e.g., disk memory) for the storage of .diagnostic routines,In addition, the width of each memory word should be sufficient to supportBIT requirements:L fn control memory to achieve controllability of herdwarecomponents.2. fn main and secondary memory to provide for error detection anderror correction techniques, as required.50●

MU-STD-2165APPENDIX A@Finalty, it is important that a sufficient number of mernary words exea=igned to non-alterable memory resources (e.g., read only memory, protected memoryareas) to ensure the integrity of critical test routinea end data. Sufficient hardwere endsoft ware redundancy should exist to confidently load criticat software segments.●b. Application software test provisions. The system epptication software(mission sof twar~ should provide for timely detection of hardware faults. Tireapelica tion SOftw~e design should includa sufficient interrupt and trap capability toSupport the immed]ate processing of errors detected by concurrent BIT hardwere prior tothe destruction of data bases or loss of information concerning the nature of the error.The operating system and each critical application program must contain softwarechecks sufficient to meet failure latency requirements.Application software error processing. Srror processing routines in theapplication ;of tware invoked by interrupts and trap should be designed with the fultparticipation of hardwere design end test engineers. The processing to be performed (rentry,automat ic error correction, diagnostic catl, operator message, error logging,immediate halt, etc.) must be consistent with the failure modes and effects analysis.‘fire operating system hierarchy should be designed to atlow the diagnostic softwaresufficient control and observation of herdware components.50.6.8 System-level Built-in test. System BIT includes a mix of BIT hardwere, BITsaftware, mnd application software error checks to provide the required degree of faultdetection and isolation at the system level.BIT intermittent failure detection. System BfT must be designed torespond ina’a predictable manner to intermittent failures, considering both themaximizing of safety end the minimizing of BtT aterms. Detection of a failure by BtTshould be followed by a second test of the failing operation, whenever practical. Thenumbers of repeated tests and repeated failures necessary to estabtish a sotid faultcondition needs to be determined. Conditions under which the operator is to be notifiedof recoverable intermittent failures should be determined. based uoon failure criticality.frequency of concurrence, and trends. For digital systems, failure &ata may be reco~d~din a failure h~tory queue. Data from the failure history queue could be made accessibleIto assist in troubleshooting of intermittent failures and to identify hardware which istfending toward solid failure. The initial implementation of system BIT shoutd be\ flexible. For example, test tolerances may be stored in software or firmware so that the1. tolerances and filtering algorithms may be easily changed if BIT is generating too manyfalse alarms.i1. b. BtT failure location. Suitable BfT features must be incorporated intoI the system to localize failures to a small number of items and to advise operatorpersomel of degraded mode options. In some cases, the BIT may need to isolate a failureIto a level lower than a replaceable item in order to determine what system functions arelost and which system functions are operational. When subsystems are to be developedI,by subcontractors, each subsystem specification should contain a requirement for selfcontainedtest with minimal reliance upon the system contractor to perform detailedLtesting of each subsystem through system-level test. The interface between system endI‘o51

IMIL-61’W2165APPENDIX Asubsystem test should be straightforward and relatively simple (e.g., test initiate, test ●response, test signature). This allows for the evaluation end demonstration of BITquality for each subcontractor prior to system integration.c. Fault-tolerant design coordination. If system availability or safetyrequires continued system operation in the presence of certain faults, then the faulttolerantdesign and testability design efforts should be closely coordinated. Equipmentredundancy or functional redundancy may be used to assist in testing. Fault esseasment, “reconfiguration into degraded mode, and configuration verification should makemaxi mum use of testing resources. The design should provide for the independent testingof redundant circuitry.50.6.9 Application of testability measures. Testability achievement is trackedthrough sys~ce use utilizing the measures listed inTable fIf, or similar measures, as appropriate to the phase of system development. TableIfI provides general guidance on applicability of measures and is subdivided into threebasic areas:a. fnherent (design) measures. fnherent testability measures areevaluations of testability dependent only upon item design characteristics. Theevaluation identifies the presence or absence of hardware features which support testingand identifies general problem areas. The analysis primarily serves as feedback to theerf or ming activity at a point in time when the design can be changed relatively easily.?See 50.6.10 and 50.6.11)b. Test effectiveness measures. Test effectiveness measures areevaluations of testability dependent upon item design, its relationship to the chosen ●maintenance environment and the testing capability of that environment. (See 5.7.3-.5)fn-service measures. fn-eervice testability measures are evaluations oftestability ~~ed upon measurable field (i.e., operational) experience. (See 5.7.6)5.6.10 Qualitative inherent testability y evaluation. Subtask 202.2.3 requires that theperforming activity gives early visibility to teatability issues and shows, in a qualitativemanner, that the testability considerations have been included in the preliminary design.Testability considerations include, as a minimum, those concepts described in 5.6.4through 5.6.7.50.6.11 fnherent testability assessment. Subtask 202.2.3 requires that the inherentability of en Item to support high qualltY testi~ be assessed using Appendix B./50.6.11.1 Preliminary design activities. The performing activity prepares e checklistof testability-related design issues which are relevant to the design, and proposes amethod of quantitatively scoring the degree of achievement of testability for eachdesign issue. A checklist is to be prepared for each item specified by the requiringIauthority. Each checkfist is tailored to the design characteristics of the item. Theprocuring authority reviews the checkliit and the scoring criteria and either concurs ornegotiates changes. The checklist approach allows for “structured” testability design152

MtkS1’D-2165APPENDIX A0Tatrle DI. TeatabLfity Measure ApplicationGuidance Matrixkstatrttlty Prefinltnery Detatl ProductioIdMeasure —Es!laL !?s93!! ~“~:”tio” Deploymentnherent x x‘eat EffectivenessFunctional coverage x xPredicted FD/Fl xpredicted FD/FI times xPredicted test coat x xxxxx=erviceAchieved FD/FIAchieved FI timeFA/CND/RTOK ratesActual test coatxxxxFDIFI Fault detection and fault isolationFA/CND/RTOK False alarm, cannot duplicate and retest okay53

MIL-STD-2165APPENDIX Aapproaches, such ss scan path, to recei~e 8 high score simply because of their inclusion ●In the dewgn. The checklist should be fmallzed prior to the preliminary design review.50.6.11.2 Checklist scoring. As the design progresses, each checklist issue is examinedand scorsd for testability compliance. The use of objective, automated testabilityanalysis tools (e.g., SCOAP, STAMP) for scoring is permitted. The scores are weightedand summed, giving a single inherent testability figure of merit for the design. Thedesign for testability process continues until the figure of merit reaches a predetermined “threshold value.50.6.11.3 Threshold determination. The requiring authority must assign a thresholdvalue to be used for the inherent testability assessment. Due to the wide variety ofpossible items which may be analyzed, a single l~bestl! threshold value cannot berecommended although a value in the range of 80 to 90 should force proper attention tothe assessment process. The actual value chosen is not alf that criticef since anotherdegree of freedom is available through the negotiation of weighting factors after thethreshold is established. In fact, since each checklist issue is weighted according to itsperceived importance in achieving a testable design, both the eventuet design and theeventual meeting of the overall figure of merit criteria are essentially determined by therequiring authority concurrence on the issues to be included and the scoring for eachissue. It is incumbent upon the requiring authority to be aware of the importance ofeach proposed issue in achieving a testable design.50.7 Task 203- Testability detail design and analysis50.7.1 Testability design techniques. During detail design, the testability designtechniques of the preliminary design are further refined and implemented. Guidance ●provided in 50.6.3 through 50.6.8 for preliminary design applies equally to detail design.50.7.2 Inherent testability y assessment. During detail design, Appendix B may beapplied to the evolving design. Guidance provided in 50.6.10 and 50.6.11 for preliminarydesign applies equally to detail design. The inherent testability assessment should becompleted prior to the critical design review.I(50.7.3 Test ef fectiveneas messures. At the completion of system or equipmentdesign, test sequences should be generated for that design and test effectivenessmeasured. Analysis need not wait for the completion of TPSS or BIT software. The use .of models (50.7.4) is encouraged since they can analyze test effectiveness on a largenumber of postulated faults (approaching 1-00%of the-specified failure population) pri~rto incorporating the test stimulus in a teat language (e.g., ATLAS) or embedded .computer language. The results of the analysis can feed forward to influence TPS or BITsoftware design and feed back to influence redesign of the prime item to improve itstestability. The identification of undetected or poorly isolated failuras can lead to threeactions:a. The failure is not detectable by any test sequence. Any f?ilures, suchas those in unused circuitry, which are impossible to detect are deleted I !from the failure population.54

MIL-STD-2165APPENDIX Ab. The failure is potentiatty detectable, but the test sequence is deficient.●Additionat stimulus patterru ere added to the test sequence.c. The failure is potentially detectable, but the item’s herdwere designprecludes the use of e test sequence of reasonable length or complexity.The prime item is redesigned to provide additional test control, testaccess, or both.“ Test effectiveness measures include functional coverage (an enumeration of whichfunctions in en item are exercised by a test), failure-based measures es described below,-end cost or benefit measures (50.7.5).50.7.3.1 Fault coverage. Fault detection coverage (FD) is cetculeted es foltows:FD= Ad whereTd= K Li,4,=where ~ i is the failure rate of the ith detected failure, ~ is the overallfailure rate, and K is the number of detected failures.o50.7.3.2 Fault resolution. To calculate predicted fault resolution, date are requiredwhich corretate each detected failure with the signature it produces during testing. ‘lIredata are mast conveniently ordered by signature end by failed module within eachsignature (fault dictionary format).N= number of unique signatures in dictionary (number of unique testresponses)signatureindexnumber of modules iisted in signatureimodule index within signaturefailure rate for jth module for failures providing signatureiOVWUI failure rate of detected failures =N M’Z, g ‘ijFauit Resolution to L or fewer modules (% replacements withambiguity S L)55

MJIA3TD-2165APPENDIX AIf alf modules under a signature are replaced es a block:FRL . ~ $ Xi ~~ij1 if MiS L‘here ‘i = { Oif Mi >Lff detailed failure rate information is unavailable or unreliable, or it is desired toconsider each fault as having equal probability, the above equation reduces to:100 NFRL = — XiMiK i= 5 .where K = number of detectedfaults = if MiIf each of the L modules under the signature group is replaced, in turn, and the test rerunfor PASS or FAIL:50.7.3.3 Fault detection time. Fault detection time (or failure latency) is the timewhich elapses between the occurrence of a fault and the detection (reporting) of thefault by the test process. This measure is most useful in characterizing how BIT deafewith critical failures. A suggested format is shown in the following example:% of Class Max. detection time●Failure class 1(most critical)95% ~ 1 second{ 100% 51 minuteFailure class 2 (critical) 85% 51 minuteThis measure requires the enumeration of signals considered most critical, critical, andso forth and an estimation made of the worst case failure latency for each signaLS0.7.3.4 Fault isolation time. During maintenance actions using BIT or off-line test,the time to fault isolate is often the largest and most unpredictable element of repairtime. The testability program should not only attempt to reduce the fault isolation timebut should also try to provide accurate predictions of fault isolation times tomaintenance planners per Task 205 of MIL-STD-470A. The fault isolation time may beexpressed es en average time, a maximum time (at some percentile), or both. The timeis based not only on the length of the diagnostic test sequence but also must include enestimation of time required for any manual intervention (e.g., the use of a sequentialsubstitution fault isolation procedure).5;”7”4 P=’=The physical insertion of a sufficient number of faults intoen Item to eterm me lts response to a test sequence has some obvious problems. Thetwo most serious problems are that the time end expense of inserting even a smallnumber of representative faults into the item is prohibitive and the ability to insert56●1(II

MIL-BTW2165APPENDIX A● representative faults is timited by circuit packagin A computer program may be usedto inject (through softwere) a large number of raults into a software model of thehardware item. The same program can simulate the behavior of an item containing oneof the faults in resoonse . . to the stimulus, The test stimulus mav then be evaluated interms of fault detection and fault resolution based upon a large fiumber of fault cases.Computer programs are wetl established es toots to simulate the fault behavior of dtgitalcircuits and grade digitet teats for Test Program Set development. These same programsmay be used to grade built-in tests using bootstrap sequences, microdiegnostics, BITsoftware patterns, etc. es test stimulus. in addition, several programs automaticettygenerate test sequences for the simulator to grade. An example of this is the- Hiererchicel Interactive Test Simulator (HtTS) program. Programs are etso available tosimulate the fault behavior of enatog circuits, but the test stimuti must be providedmenuetty. The usefulness of this approach depends upon the ability of the modets (itemmodeta end fault modefs) to accurately reflect actual operation end actual field failures.‘Me item must be modeled at a level of detail which aUows elt important failure modesto be included. The fault-free behavior of the item model must be validated orior tomodeling faults by applying a functioned test and comparing the modeled respo~ to theexpected response or to the response of a known good hardware item.50.7.5 System level test effect iveness. The level of BIT fault detection for theoverall system is calculet ed av.~D . ~ 1 iFDi~~ii = 1 to number of itemswhere i is the failure rate of the ith item end FDi is the fault detection prediction forthe i th item. This applies equally for systems with centralized BIT and systems withdistributed BIT (i.e., BtT in each item).50.7.6 Testability cost end benefit data. Ultimately, aU test performance measurestranslate into cost impacts. Higher quality tests usuelty cost more to produce but shouldresult in cost savings over the tife cycle of the system. These cost data are critical insetting reasonable testability requirements within the framework of supportabilityanalysis. Subtesk 203.2.7 ensures that testability cost data for this acquisition programare available for incorporation into appropriate data bases for use by futuresupportability analysis efforts.a. Non-recurring costs. The development costs associated with theincorporation of testability mto the system or equipment include, but are not fimited to,the fotlowing:Testabitit y program planning costs;: Testability design costs3. Testab]tity modeling end anetysis ccmts4. Testability data preparation costs.b. Recurring COStS and penalties. The producti~n, operation endmaintenance costs and penalties associated with the incorporation of t~tability into thesystem or equipment include, but are not limited to, the following:57

~..——— ——-—1.2.3.4.5.6.MIL-BTD-2165APPENDIX APer item costs of additional herdware required for BIT endtestability capabilities.Volume and weight required for additional hardwere, additional●connectors, and increased modularity.Power required for additional hardware.ComputerPossibilitymemory required for BIT software.of system interruption due to failure within BIT -circuitry.Refinability impact due to additional herdwere.c. Benefit assessment, development and production. The impact oftestability on development and production costs includes, but is not limited to, thefollowi~1. Test generation costs2. Production test costs3. Test equipment costs4. fnterface device costsd. Benefit assessment, operation and maintenance. The impact (actual orpredicted) of testability on operation and maintenance COStSincludes, but is not Iimitedto, the foflowing:L Teat and repair costs2. Test end repair time3.4.Manpower costsTraining costs5. Spares cost50.7.7 In-service testability measures. in-service testability measures evaluate theimpact of actual operational and maintenance environments on the ability of productionsystems and equipments to be tested. The planning for the cof.lection of data to measuretest effectiveness in the operational and maintenance environment is accomplishedthrough Task 103. It is important to realize that testing problems in the field may becorrected in many ways (e.g., personnel changes, organizational changes, proceduralchanges, etc.) and do not always result in engineering design changes. fn-servicetestability measures include:a. Level of automation. Are the testing tools provided consistent with thetraining/skU levels of assigned personnel?b. BIT fault detection. Does BfT provide timely and accurate detection offaults so as to minimize reliance on manual detection (e.g., squawks)?c. BIT false alarm rate. Are BIT false alarms adversely impactingoperational availability and maintenance workloads?●(

MU-BTD-2165APPENDDZ Aod.e.Retest okay. Are feults detected at one level of maintenance etsodetected at the nest level of maintenance?BIT feult isolation time. Does BIT support system MITR and systemavaifabitity requirements?“.f.g,h.Off-tine fault isolation time. Does ATE end its associated TPS6 supportshop throughput requirements?Fault resolution. Does poor fault resolution for BIT or ATE adverselyimpact spares availabttity?BIT reliability. fs poor BIT reliability adversely impacting the mission?50.8 Teak 301- Testability demonstration50.8.1 Demonstration parameters. It is useful to distinguish between fault detectionand isolation at the system level (organisational maintenance Ievet) and fault detectionand isolation performed off -line (higher maintenance levels). The former may bedemonstrated as part of a standard maintainability demonstration if certain testabilityconcerns (Subtask 301.2.1) are incorporated into the maintainability demonstration plansand procedures. The latter may be.demonstrated as part of the evaluation proceduresfor Test Program Sets, including evaluation of softwere, interfaces and documentation.50.8.2 BfT and off-tine test correlation. Through the development of the testabilitydemonstration plan, the items to be demonstrated under the maintainabilitydemonstration end the TPS demonstration may be coordinated (e.g., some common faultsto be inserted) so es to provide data on the correlation of BIT resutts and off-tine testresults. This can give an early indication of possible “cannot dupticate” (CND) problemsin the field.50.8.3 BIT false alarm rate. One important testabtiity parameter, BIT false etarmrate, is difficutt to measure m the controlled environment of a demonstration procedure.If the fatae alarm rate was relatively high, it would be possible to make use of areliability demonstration procedure from MIL-STD-781 to demonstrate the false alarmrate, treating each BIT false alarm es a relevant failure. The environmental conditionsduring the demonstration should be indicative of the expected operationet environment inorder to experience a wide range of false alarm causes.50.8.4 Model vetidation. Even with a reasonably large sample of inserted faults, ademonstration can yield only timited data on actuat test effectiveness. However, ademonstration is also useful in validating some of the assumptions end modets that wereused during the earlier testability analysis and prediction efforts (Task 203) which werebased upon a much targer fault set. If certain assumptions or models are invaUdated bythe demonstration, appropriate portions of Task 203 should be repeated and newpredictions should be made.59

MIL-STD-Z165APPENDIX A10. SCOPE10.1 -e. This appendix provides requirements for the assessment of theinherent testablhty of system or equipment design.10.2 Application. Appendix B shell be considered es forming a pert of thestandard.20. REFERENCED DOCUMENTSNot applicable.30. DEFINITfONSNot applicable.40. GENERAL REQUIEEMENTN40.1 General. Conduct an analysis of the inherent (intrinsic) testability of thedesign. The analysis identifies the presence or absence of hardware features whichsupport testing and identifies problem areas. The method of this Appendix shalt beaPplied to each item identified for fnherent Testability Assessment by the requiringauthority. The data required by this appendix shalI be determined by the performingactivity and approved by the requiring authority. Any testability criteria designated asmandatory by the requiring authority, and therefore not subject to design tradeoffs,should be assessed separately from this procedure.50. DETAILED REQUIREMEN’R3●50.1 Procedure. Assess the inherent testability of the system or equipment designusing the frherent Testability Checklist, Table IV.to the desig~.Delete those testability criteria from Table IV which are not applicableb. Add additional testability y criteria to Table IV which are relevant to thedesign (or modify original criteria).c. Assign weighting factors (WT) to each item based upon its relativeimportance in achieving a testable product. (1~ WTA1 O)d. Develop a scoring system for item (OSscores100) where 100 represents “maximum testability and Orepresents a complete lack of testability.authority.e. Obtain concurrence on (a) through (d) above from the requiring60

’MII.AWD-2165APPENDIX ACount the desi~ attributes which are relevant to each tastabitity item(e.g., the to!el number of nodes m a circuit).Count the design attributes which meet the testeb~ty criteria for eachitem (e.g., ~~e number of nodes accessible to the tester).total nodes,grApply the scoring system to each item (e.g., Score = accessibleScore = 100 if YES end = Oif NO).nodes+.i. Calculate the weighted score for each item, WT Score = WT x Score.Cslculate the inherent testabLtity of the design, TESTABILITY = Sum(wT Score)! Sum (WT).“50.2 Criteria. Modify the de@gn as necessary until the inherent testability equalsor exceeds the threshold value.

Table IV.MIL-BTD-2165APPEWDIX BInherent TakabilityChecklist●Mechanical DesignIS a standard grid layout used on boards tofacilitate identification of components?WTTotalNumberNumberMeetingCriterie*oreWTBcore✟✍✎Is enough spacing provided between componentsto allow for clips and test probe?Are elf components oriented in the samedirection (pin 1 always in same position)?Are standard connector pin positions usedfor power, ground, clock, test, etc.,signafs ?Are thepins innumber of input end outputan edge connector or(1/0)cableconnectorcapabilitiescompatibleof thewith theselected1/0testequipment?●Are connector pins arranged such that theshorting of physically adjacent pins willcause minimum damage?fs defeatable keying used on each board soes to reduce the number of uniqueinterface adapters required?When possible,included in theare1/0power end groundconnector or testconnector?✌Have test and repair requirements impacteddecisions on con formaf coating?Is the design free ofrequirements (e.g., speciafspecialcooling)set-ypwhichwould slow testing?62

MIL-STD-2165APPENDIX B●●NumbesTotal Meet@WT Number Criterin 6core 2Mechanical Design (contkt)Does the item warm up in a reasonableamount of time?IS each hardwere component clearlyIabelled?partitiord~Is each function to be tested placed whollyupon one board?If more then one function ts ptaced on aboard, can each be tested independently?Within a function, can complex digital andanelcg circuitry be teated independently?Within a function, is the size of each blockof circuitry to be tssted smetl enough foreconom icel fault detection and isolation?If required, are pult up reststora located onsame board as the driving component?Are analog circuits partitioned by frequencyto ease tester competibitlty?fs the number of ~wer supplies requiredcompatible with the test equipment?Is the number and type of stimuli requiredcompatible with the test equipment?Are elements which are included in anambiguity group placed in the samepackage?Ttst ControlAre unused connector pins used to providetest stimulus andto internal nodes? control from the- tester63

MIL-6TEM165APPENDIX BWITotalNumberMeet@criteriaScoreWTWore●Test Control (contkl)Can circuitry be quickly and easily drivento a known initial state? (e.g., masterclear, less than N clocks for initializationsequence)?Are redundant elements in design capableof being independently tested?Is it possible to disable on-board oscillatorsand drive all logic using a tester clock?Can long counter chains be broken intosmaller segments in test mode with eachsegment under tester control?Can the tester electrically partition theitem into smaller independent, easy-to-testsegments? (e.g., placing tri+tate elementsin a high impedance state).Is circuitry provided to by-pass any hmavoidable)one-shot circuitry?●Can feedback 100PS be broken under controlof the tester?In microprocessor-based systems, does thetester have access to the data bus, addressbus and important control lines?Are teat control points included at thosenodes which have high fan-in (i.e., testbottlenecks)?Are input buffers provided for these controlpoint signals with high drive capabilityrequirement9?Are active components, such as demultiplexersand shift registers, used to allowthe tester to control necessary internalnodes using available input pins?—64●

MIL-SITH165APPENDtx B‘t’em Access—WT—TbttiNumbMeeticritel—Scol—2.Are unused connector pins used to prov[deadditional internal node data to the tester?Are signal lines end test points designed todrive the capacitive loadlng represented bythe test equipment ?Are test points provided such that thetester can monitor and evnchronize toonboerd clock circuits? “Are test access points pieced at thosenodes which have high fen-out ?Are buffers employed when the test pointis a latch end susceptible to reflections?Are buffers or dividar circuits employed toprotect those test points which may bedamaged by en inadvertent short circuit?Are active components, such as muitiplexemend shift registers, used to makenecessary internal node test date avatlebleto the tester over available output pine?Are ail high voltages scaled down withinthe item prior to providbrg test pointaccess so es to be consistent with testercapabilities?Is the measurement accuracy of the testequipment adequate compared to the toierencerequirement of the item being tested?PertB SekctiOnIs the number of different part types theminimum possibie?65

Perte Belection (eont%t)Have parts been selected which are wellcharacterized in terms of failure modes?MIL-2TD-2165APPENDfX B1IWTTotalNumber-NumbsMeet@CriterfiBccmWTBcore●✌Are the parts independent of refreshrequirements? U not, are dynamic devicessupported by sufficient clocking duringtesting?fs a single logic family being used? U not,is a common signal level used forinterconnections?Analcg DesignLa one test point per dwcrete active stagebrought out to the connactor?fs eech test point adequately buffered orisolated from the main signal path?Are multiple, interactive adjust ments prohibitedfor production items?●Are functional circuits of low complexity?(Amplifiers, regulators, etc.)Are circuits functionally complete withoutbias networks or loads on some other UUT?fs a minimum number of multiple pheeerelatedor timing-related stimulusrequired?Is a minimum numbermeasurements required?of phase or timingIs a minimum number of complex modulationor unique timing patterns required?Are stimulus frequencies compatible withtester capabilities?66●

MIL-SIIF2165APtwNINX BNumber‘Ibtel Meet@ WTWT Number Crtteria score ScoreI- Desii (contkt)Are stimulus rise time or pulse widthrequirements compatible with testercapabilities?Does response measurements involve frequenciescompatible with tester capabilities?Are response rise time or putse widti measurementscompatible with tester capabitltie9?Are stimulus amptituda requirements withinthe capability of the test equipment?●DoesAre response amplitude measurementswithin the capability of the test equipment?the design avoid externet feedbackloops?Does the design avoid or compensate fortemperature sensitive components?Does the design eltow testingsinks?without heatAre standard types of connectors used?Di@atDe@Does the design contain only synchronouslogic?Are all clocks of differing phases and frequenciesderived from a single masterclock?Are etl memory elements clocked by a derivativeof the master clock? (Avoid alementsclocked by data from otherelements.)67

MIL-STD-2165APPIfNDJX BW’1TotalNumberNumberMeetingcriteriaSooreWTScore●Digital De9ign (cont4f)Does the design avoid resistance capacitanceon~hots and dependence upon logicdelays to generate timing pulses?Does the design support testing of “bitSlic,esl’?Does the design include data wraparoundcircuitry at major interfaces?Do all buses have a default value whenunselected?For multilayer boards, is the layout of eachmajor bus such that current probes or othertechniques may be used for fault isolationbeyond the node?Is a known output defined for every word ina Reed Only Memory (ROM)? Will theimproper selection of an unused addressresult in a well defined error state?●Is the number of fan-outs for each internalcircuit limited to a predetermined value?fs the number of fan-outs for each boardoutput limited to a predetermined value?Are latches provided at the inputs to aboard in those cases where tester inputskew could be a problem?fs the design free of WIRED-ORS?Does the design include current limiters toprevent domino effect failures?If the design incorporates a structuredtestability design technique (e.g., ScanPath, Signature Analysis), are afl the dasignrules satisfied?66●

MU.-STD-2165APPENDIX BWTTOMNumberNumberMee.tirgcriteria-scoreSC%Digital De3ign (contkt)Are sockets provided for microprocessorsand other comples components?Built-fnTest (BIT)Can BIT in each item be exercised undercontrol of the test equipment?IS the Test Program Set designed to tekeadvantage of BIT capabilities?Are on-bard BIT indicators used forimportant functions? Are BIT indicatorsdesigned such that a BIT fellure witt give aFAIL indication?Does the BtT use a building-block approach(e.g., all inputs to a function are verifiedbefore that function is tested)?Does building-block BIT make maximumuse of mission circuitry?fs BfT optimetly altocated in hardware,soft were and firm were?Does o-rd ROM contain setf-test routines?.D- BfT include a method of saving ontinetest data for the analysis of intermittentfailures and operational failures whichare non-repeatable in the maintenanceenvironment ?Is the failure rate contribution of the BfTcircuitry within stated comtraints?Is the additional wefght attributed to BITwithin stated constraints?Is the additional volume attributed to BtTwithin stated cottstrainte?‘... .,,(69

“”. ‘“MU-BTD-2165APPENDIX BWTTotalNumberNumberMeetiiCrReriaBcoreWTBcore●Built-inTest’(Bti)(contkf)Is the additional power consumption attributedto BIT within stated constraints?❆Is the additional pert count due to BITwithin stated constraints?Does the allocation of BIT capability toeach item reflect the relative failure rateof the items end the criticality of theitems’ functions?Are BIT ~hreshold values, which mayrequire changing es a result of operationalexperience, incorporated in software oreasily-modified firmware?Is processing or filtering of BIT sensor dataperformed to minimize BIT false alarms?Are the data provided by BIT tailored tothe differing needs of the system operatorand the system maintainer ?●iIs sufficient memory allocated for confidencetests and diagnostic softwere?Does mission software include sufficienthardware error detection capability?J?, the failure latency associated with a,psrticular implementation of BIT coneistntwith the criticality of the functionmonitored?IT threshold limits for each parame-“ned es a result of considering% ter’s distribution statistics,. urement error and the\ tection/falae alarm cher-✎—70●

MIL-BlTF2165APPENDIX BTest RequirementsWI—TotalNumbetNumbMeet@CritetiHes Level of Repair Analysis been accomplished?For each maintenance level, hes a decisionbeen made for each item on how built-intest, automatic test equipment end generalpurpcse electronic test equipment wittsupport fault detection and isolation?r)Is the planned degree of test automationconsistent with the capabilities of themaintenance technician?For each item, does the ptenned degree oftestability design support the level ofrepair, test mix, and degree of automationdecisions?Are the test tolerances established for BITconsistent with those established for higherlevel maintenance test?Teat DataDo state diagrams for sequential circuitsidentify invalid sequences and indeterminateoutputs?If a Computer-Aided Design system is usedfor design, does the CAD data base effectivelysupport the test generation processand test evaluation process?For Lerge ScaIetntegrated Circuits usedinthedesign, aredataavaitebla to accuratelymodel the LSIC and generate highwonfidencetesta for it?—71

Teat Date (cont4f)For computer-assisted test generation, isthe available software sufficient in termsof program capacity, fault modeling,component libraries, and post-processing oftest respomse data?MIL-STD-2165APPENDIX BWTTotaltumberNumberMeet@CriteriascoreWTtire●✌✎✎Are testabtlty features included by thesystem designer documented in the TRD interms of purpose and rationale for thebenefit of the test designer?fs a mechanism available to coordinateconfiguration changes with test personnelin a timely manner?Are test diagrams included for each majortest? k+ the diagram limited to a smallnumber of sheets? Are inter%heet connectiOIISclearly marked?Is the toleranceon the item?bend known for each signaf●☛—72

10.1 Appendix C shett be considered es forming a part of the tile standard.\10.2 ‘l%e purpose of this appendix is to provide definitions of terms used forclarity of understanding and completeness of information. Aa a generat rule, thedefinitions provided are currently accepted end have been extracted verbatim from otherdirectives (regulations,terms are presentedmanuals, MlL-STD%, DOD Directives, etc.). A limitedfor which definitions were developed from severalnumber ofreferencedocuments.%0. DEFINITtONBAcqutaition phases(a) Concept Exploration Phase - The identification end exploration of alternativesolutions or solution concepts to eatisfi a validated need.(b) Demonstration and Validation Phasa - The period when selected candidatesolutions are refined through extensive study and analyses; hardware development, ifappropriate; test; and evaluations.(c) FuI1-2cate Development Phase - The period when the system end the principalitems necessary for its support are designed, fabricated, tested, and evaluated.(d) Production and Deployment Phase - ‘I%e period from production epprovatuntit the last system is delivered end accepted.Built-in test (BIT). An integral capability of the mission system or equipment whichprovides an automated test capability to detect, diagnose or isolate failures.Built-in teat equip ment (BITE). Herdware which is identifiable es performing thebum-m trst function; a subset o~lT..cammt duplicate (CND). A fault indicated by BIT or other monitoring circuitry whichcannot be confirmed at tli6 first level of maintenance.Fellura tateney. me elapsed time between fault occurrence end failure indication... Fetac alarm. A fault indicated by BIT or other monitoring circuitry where no faultexists.Fault coverage , fautt detection. The ratio of failures detected (by a test program ortest procedure) to failure popule tion, expressed es a percentage.Fault iaoletton time. ‘l%eetapsed time between the detection end isolation of a fault;a component of. repair time.73

MIL-2TD-2165APPRNDIX CFault resolutiooj fault isolation. The degree to which a test program or procedure canisolate a fault within an item; generally expressed as the percent of the cases for whichthe isolation procedure results in a given ambiguity group size.●Snherem temnbility. A testability measure which is dependent only upon hardwaredesign and is independent of test stimulus and response data. 4fnterface device (ID). provides mechanical and electrical connections and any signalconditioning required between the automatic test equipment (ATE) and the unit undertest (UUT); also known as an interface test adapter or interface adapter unit.,Item. A generic term which may represent a system, subsystem, equipment, assembly,su=mbly, etc., depending upon its designation in each task. Items may includeconfiguration items and assemblies designated as Units Under Test.Off-line ttii~. The testing of an item with the item removed from its normaloperational environment.Performi~ activity. That activity (government, contractor, subcontractor, or vendor)which is responsible for performance of testability tasks or subtasks as specified in acontract or other formal document of agreement.Requiri ng authority. That activity (government, contractor, or subcontractor) whichlevies testa~llity task or subtesk performance requirements on another activity(performing activity) through a contract or other document of agreement.●opining,A unit underbut performstestthatthat malfunctions in a specific manner duringspecific function satisfactorily at a higher levelmaintenance facility.‘C~li~ A (fes”lgn characteristic which allows the status (operable, inoperable, ordegra e o an item to be determined and the isolation of faults within the item to beperformed in a timely manner.T@ effectiveness. Measures which include consideration of hardware design, BfTdesign, test equipment design, and t=t program set (TPS) design. Test effectivenessmeasures include, but are not limited to, fault coverage, fault resolution, fault detectiontime, fault isolation time, and false alarm rate..Test program set (TPS). The combinationprogram instruction, and supplementary dataof testrequiredprogram, interface device, testto initiate and execute a given.test of a Unit Under Test.Test reqmr . ements document. An item specification that contains the requiredperformance characteristics of a UUT and specifies the test conditions, values (andaflowable tolerances) of the stimuli, and associated responses needed to indicate aproperly operating UUT.74 ●

●lNSTRUC170NSILI. mrI!k&s cffmtta nuka our standardization documenu battar,UIe DoDpvvidu M form for we incubmk!@commmtiand LIUSUtkmt for Imsuonmcnta M um of mOifuY dadardhtion dommenu am kited h EUVvide9USSCLUOI!U ‘f?h form IBSY b dctacfm!.foldeddons II!*line!indicated,Uti dons IfIo100WM@(~ NOYSTAPLE),andmdhd. Enbtock6, bt u spad!k u uaalbte clout puitcuhr problem - mch u wrdhs did reQuimd Mcrpreimkort. -LOO rid. ~-. h. -M--- = - komd~*. d sfm Pwowd --dins CIWISea wbicb wouldSU*W tie~blenm. EnterInblack6 my mnuzh mt mhmdto c ●DRCI!EC ~gb of the document.U block7 Is Cwd out, anukn0ddS911E01 d] b dd b YOU -ftbin SO &y# b tat YOU know Lbd YOW cOrnmell tswm, r+mtredauduahhtgmaddemd.NOTE: ‘fhkIform my not b, d t-a ?8~U~ c@ci of documents,no, to requealwdmm,dmhtiom,o, cluifkitkm ofmadnmfianIUEulmaeation mat eantmc!s.Commenttsubmitted00 tht8formdo DOIcorudmtaLUIIBD!Yautboriamic.n-1. to walm my pottk.m of tbo”ufermmd document(,) or to .merid e.ontrutud requlmumifs.t(Fold OIwu Ihb Uml]. .“(Fbld do”, 1M9lIIW,&DEPARTMENTOF THE NAVVNaval Electronic SystemsWashington, DC 20363OFFtCIAL 8USINE=. ?ENAI.TV FOR ●R,VATS uSC SS00FIRSTBUSINE:SNoR~PLYMAILC1..ss WAS M! NGTON 0, C,PCSIAOE WILL 8E PAID BY THE DEPARTMENT OF THE NAVYCW’4ANDERNAVALELECTRONIC SYSTEMSCONMANDDEFENSE STANDARDIZATION PROGRAM BRANCHDEPARTMENT OF THE NAVYWASHINGTON , OC 20363ATTN : ELEX 8111111111nNO ?OSTAON6cE6s*nUNITE!!=1IF MAILEDIN TMESTATESEYI

IIIIII~ANDARDIZATION OOCUMENT IMPROVEMENT PROPOSAL(SeeInstructimu- Rfvaie SW)OCUMENT NJMBER 2. ~cuM~NT ‘lTLE TESTABILITYPROGRAN POR ELECTRONIC SYSTEMS.-STD-2165VAME OF SUBMITTINOOfl QANIZATlON•1VENOOll●I1.I,DOnE’6s@hwLIX*. Srati, ZIP Code)❑ &MANuFAC7UI!ER❑ ’a C.T14E14,epdh): “,IIIROSLEM AREASP.rDJrQIII N“ mbor and WordlwIb. ROconlrm”ti Wo,dmg ,●c FI-n/ R., ion.!. for R_mrnondmi.m:III3EMA13KS4..I.IIIIIIIIIIINAME OF S“9M,TTE~ & 1, Ph t. ml) - or,,londb, WORK TE LEPHONe NUMBER (1”4”- Arcscd) - OptlolulUAILING ~~0~gS9,.sfm.1, C,fy, 810t8. ZIP Cd) - Oc.tle”d @. DATE OF SU8MI6S1ON O’YMMDD)mm .— —.. . -&-w &“J:.1426PREv1OU9 EOITlON m qSOLETE.●

L!EE4P.!E!!mMI L-STD-2165ANOTICE 131 July 1995 “MILITARYSTANDARD?ESTABI L ITY PRDGRAM FOR SYSTEMS AND EQUIPMENTSMI L-STD-2165A, dated 01 February 1993, is hereby canceled and superseded byMI L-HDBK-2165, Testability Handbook for Systems and Equipment.Custodians: Preparing activity:Army - CR Navy - ASNavy - SliAir Force - 17 (Project ATTS-0001)AMSC No. N6742 AREA ATTSDISTRIBUTION STATEMENT @. Approved for public release; distribution is unl imited.