Automated Generation of Failure Modes and Effects Analyses from ...

Automated Generation of FailureModes and Effects Analyses fromAADL Architectural and ErrorModelsMyron Hecht, Alexander Lam, Russell Howes, and Christopher Vogl,The Aerospace CorporationPresented toSystems and Software Technology ConferenceSalt Lake, City, UTApril, 2010© The Aerospace Corporation 2010

Outline• Motivation• Background on FMEAs• Introduction to AADL• AADL Error Model Annex• Tool Set for Analyzing Risk and Reliability/Availability• Automated FMEA Generation Example• Additional Discussion• Conclusions2

Motivation• Failure Modes and Effects Analyses (and related Criticality Analyses)are rigorous and comprehensive reliability and safety designevaluations– Generally required either by industry standards or Government policies– A fundamental element of defense in many product liability lawsuits• When performed manually, FMEAs are usually done only once duringthe detailed design phase because of cost and schedule constraints– Labor intensive– Require senior level; analysts• If automated, FMEAs would have significant benefits– Multiple iterations from conceptual to detailed design– Enables early identification of potential problems• Single points of failure• Unanticipated effects– Facilitates tradeoff studies and evaluations of alternatives3

Failure Modes and Effects Analysis (FMEA)• Purpose– To determine the effect of hardware and software failures upon the systemand equipment failures.• Classify effects by impact on mission success and personnel/equipmentsafety.• Identify single points of failure• History– First defined as Military Procedure MIL-P-1629, “Procedures for Performinga Failure Mode, Effects and Criticality Analysis”, November 1949.– Further developed and applied by NASA in the 1960’s to improve and verifyreliability of space program hardware.– Since the 1980s, a standard of practice in a wide variety of industries• DoD: MIL-STD-1629A• Industrial: IEC 60812 (1985)• Aviation: SAE ARP 5580 (2001)• Automotive: SAE J1739 (2002)• Space: ECSS-Q-30-02A4

FMEA MethodologyConventionalDefine Ground Rules and AssumptionsLevels of indentureComponents to be consideredFailure modes by componentcategorySeverity Level DefinitionsRules for recovery mechanismsand compensating provisionsFor Each ComponentPostulate failure and failure modeIdentify immediate effect of failureIdentify next higher level effectsand “end effects”Identify compensating provisionsEvaluate severity level at endeffectAutomated• Ground rules and assumptionsdefined by component properties• Components and failure modesdefined in models• Effects identified throughgraph tracing5

FMEA OutputIn Either Worksheet or Tabular Format…• Identification: Failure Mode identification.• Item: For software, a process in its context.• Failure Mode:– Immediate Effect:– Intermediate Effect: Second level effect.• Operator• External networks• Database• Recovery– End Effect:• System Level (e.g., Individual satellites or the constellation through TT&C functions)• Payload performance• Data to outside users through terrestrial interfaces• Existing Mitigations: Any existing mitigations present in the architecture ordesign were identified.• Severity level:– Set under assumption that existing mitigations assumed to work• Comments:– Additional comments documenting assumptions and uncertainties.6

Introduction to the Architecture Analysis& Design Language (AADL)• Society of Automotive Engineers (SAE) Aerospace Standard AS5506(2004)– Preceded by more than a decade of development under the DARPA Meta-H program• Provides a standardized textual and graphical notation for describingsoftware and hardware system architectures and their functionalinterfaces– architectures (using standard language).– expected program behavior (using behavior annex)– Failure and recovery behavior (using error annex)7

AADL vs. other OMG Languages for Stochastic Analysisof Risk and Reliability• Advantages– Objects directly represent real-time system hardware and software– Standard method for incorporation of quantitative attributes• Failure and Recovery Probabilistic Distributions• Parameters of those distributions• Probabilities and rates for individual transitions– Standard methods for representing propagation of failures across multiplecomponents• Event ports for failure propagations• Guards to enable conditional propagations (important for abstractionsand reuse)• Drawbacks– No commercial quality tools• Public domain tools are available and usable – but not bug free8

AADL Components (graphical representation)– text and xml representations also defined9

AADL Error Model Annex• AADL annex that supports stochastic analysis• Defines error model– State transition diagram that represents normal and failed states– Error models can be associated with hardware components, softwarecomponents, connections, and “system” (composite) components• Error model consists of– State definitions– Propagations from and to other components– Probability distribution and parameter definitions– Allowed state transitions and probabilities10

Enabling Features of AADL• Standard representation of architecture and error models• Representation of failure propagation through systemcomponents– Event Ports– Guards– Propagations• Error Model properties– Working status of states– Descriptive information for initial states, effects (subsequentstates), and failure modes (transitions)– Initial states– Terminal States

AADL Error Model ExampleFailvisible (in)error model examplefeaturesErrorFree: initial error state;Failed: error state;Fail: error event {Occurrence => poisson lambda};Repair: error event {Occurrence => poisson mu};Failvisible: in out error propagation {Occurrence => fixed p};end example;error model implementation example.generaltransitionsErrorFree-[Fail]->Failed;Failed-[Repair]->ErrorFree;ErrorFree-[in Failvisible]->Failed;Failed-[out Failvisible]->Failed;end example.general;Failvisible(out), prob p12More information: Feiler (2007)

AADL Tool Set• Eclipse Development Environment (Ganymede) and Eclipse ModelingFramework (EMF)• Component plug-ins– TopCASED graphical editor to create AADL architecture diagrams (SEI,Aerospace modifications)– Error Model Editor graphical editor to create AADL error model diagrams(The Aerospace Corporation newly developed)– OSATE AADL generator (SEI, The Aerospace Corporation modifications)– ADAPT-M Stochastic Petri net to MoBIUS stochastic analysis network tool((SEI/LAAS Toulouse and The Aerospace Corporation)– MoBIUS Quantitative Dependability modeling and prediction tool(University of Illinois, Champaign Urbana)– FMEAGEN FMEA Generator (The Aerospace Corporation newlydeveloped)13

AADL Modeling Tool Chain Data FlowQualitative Analysis ChainQuantitative Analysis Chain14

Tool Set Screen Shot15

FMEA Generation Algorithm Features• Automatically traces from all working states to failure states– Terminates when trace detects a restoration condition or a failurecondition• Not limited to only 3 levels of effects• Checks to prevent repeated visits to same states– Ensures termination– Of particular importance for recoverable systems

Example: Supplemental Restraint SystemArchitecturalModelError Models

Generationof FMEAfrom PetriNet of ErrorModels

Results: Automatically Generated FMEAEnhanced formatting for presentation purposes

More ComplexError Model20

Results: Automatically Generated FMEAIDItemInitial FailureMode1st Level Effect Transition 2nd Level Effect Transition 3rd Level Effect Transition 4th Level Effect Transition 5th Level Effect1.1 SBCU.Primary_SU Failure SU.SBCU_Primary ReportDownSBCUSdown from SBCU.Primary_SU toSBCU.Primary_SUSU.SBCU_Primary DownFailure_case_Minor from SBCU.Primary_SU toSBCU.Primary_SUSU.SBCU_Primary DownMinorRecoverMinor from SBCU.Primary_SU toSBCU.Primary_SUSU.SBCU_Primary ReportRecoverSBCUSrecover from SBCU.Primary_SU toSBCU.Primary_SUSU.SBCU_Primary HotStandbySBCUSrecover from SBCU.Primary_SU toSBCU.FMSFMS.SBCU UsingPrimary1.2.1SBCU.FMS guardin PrimaryDown fromSBCU.Primary_SU to SBCU.FMSFMS.SBCU PrimaryisDown1.2.2.1Failure_case_Major from SBCU.Primary_SU toSBCU.Primary_SUSU.SBCU_Primary DownMajorRecoverMajor from SBCU.Primary_SU toSBCU.Primary_SUSU.SBCU_Primary ReportRecoverSBCUSrecover from SBCU.Primary_SU toSBCU.Primary_SUSU.SBCU_Primary HotStandby1.2.2.2SBCUSrecover from SBCU.Primary_SU toSBCU.FMSFMS.SBCU UsingPrimary1.3SBCU.FMS guardin PrimaryDown fromSBCU.Primary_SU to SBCU.FMSFMS.SBCU PrimaryisDown2.1.1 SBCU.Backup_SU Failure SU.SBCU_Backup ReportDownSBCUSdown from SBCU.Backup_SU toSBCU.Backup_SUSU.SBCU_Backup DownFailure_case_Minor from SBCU.Backup_SU toSBCU.Backup_SUSU.SBCU_Backup DownMinorRecoverMinor from SBCU.Backup_SU toSBCU.Backup_SUSU.SBCU_Backup ReportRecoverSBCUSrecover from SBCU.Backup_SU toSBCU.Backup_SUSU.SBCU_Backup HotStandby2.1.2SBCUSrecover from SBCU.Backup_SU toSBCU.FMSFMS.SBCU UsingBackup2.22.3SBCU.FMS guardin BackupDown from SBCU.Backup_SUFMS.SBCU Downto SBCU.FMSSPCU.FMS guardin BusDown from SBCU.FMS toFMS.SPCU WaitingForBusSPCU.FMS2.4SPCU.Primary_SU guardin FMSstandby from SPCU.FMSSU.SPCU_Primary ColdStandbyto SPCU.Primary_SU2.5.1Failure_case_Major from SBCU.Backup_SU toSBCU.Backup_SUSU.SBCU_Backup DownMajorRecoverMajor from SBCU.Backup_SU toSBCU.Backup_SUSU.SBCU_Backup ReportRecoverSBCUSrecover from SBCU.Backup_SU toSBCU.Backup_SUSU.SBCU_Backup HotStandby2.5.2SBCUSrecover from SBCU.Backup_SU toSBCU.FMSFMS.SBCU UsingBackup2.6SBCU.FMS guardin BackupDown from SBCU.Backup_SUFMS.SBCU Downto SBCU.FMS2.7SPCU.FMS guardin BusDown from SBCU.FMS toSPCU.FMSFMS.SPCU WaitingForBus2.8SPCU.Primary_SU guardin FMSstandby from SPCU.FMSSU.SPCU_Primary ColdStandbyto SPCU.Primary_SU3.1 SBCU.Primary_PU Failure PU.SBCU TerminatedCPUfail from SBCU.Primary_PU toSBCU.Primary_SUSU.SBCU_Primary Terminated3.2SBCU.FMS guardin PrimaryTerminated fromSBCU.Primary_SU to SBCU.FMSFMS.SBCU PrimaryisTerminated4.1 SBCU.Backup_PU Failure PU.SBCU TerminatedCPUfail from SBCU.Backup_PU toSBCU.Backup_SUSU.SBCU_Backup Terminated4.2SBCU.FMS guardin BackupTerminated fromSBCU.Backup_SU to SBCU.FMSFMS.SBCU Down4.3SPCU.FMS guardin BusDown from SBCU.FMS toSPCU.FMSFMS.SPCU WaitingForBus4.4SPCU.Primary_SU guardin FMSstandby fromSPCU.FMS to SPCU.Primary_SUSU.SPCU_Primary ColdStandby5.1 SPCU.Primary_SU Failure SU.SPCU_Primary ReportDownSPCUSdown from SPCU.Primary_SU toSPCU.Primary_SUSU.SPCU_Primary Down Recover from SPCU.Primary_SU to SPCU.Primary_SU SU.SPCU_Primary ReportRecoverSPCUSrecover from SPCU.Primary_SU toSPCU.Primary_SUSU.SPCU_Primary ColdStandbySPCUSrecover from SPCU.Primary_SU toSPCU.FMSFMS.SPCU UsingPrimary5.2SPCU.FMS guardin PrimaryDown fromSPCU.Primary_SU to SPCU.FMSFMS.SPCU Down6 SPCU.Backup_SU Failure SU.SPCU_Backup ReportDownSPCUSdown from SPCU.Backup_SU toSPCU.Backup_SUSU.SPCU_Backup Down Recover from SPCU.Backup_SU to SPCU.Backup_SU SU.SPCU_Backup ReportRecoverSPCUSrecover from SPCU.Backup_SU toSPCU.Backup_SUSU.SPCU_Backup ColdStandby7.1 SPCU.Primary_SU Failure SU.SPCU_Primary ReportDownSPCUSdown from SPCU.Primary_SU toSPCU.Primary_SUSU.SPCU_Primary Down Recover from SPCU.Primary_SU to SPCU.Primary_SU SU.SPCU_Primary ReportRecoverSPCUSrecover from SPCU.Primary_SU toSPCU.Primary_SUSU.SPCU_Primary ColdStandby7.2SPCU.FMS guardin BackupDown fromSPCU.Backup_SU to SPCU.FMSFMS.SPCU Down8.1 SPCU.Primary_PU Failure PU.SPCU TerminatedCPUfail from SPCU.Primary_PU toSPCU.Primary_SUSU.SPCU_Primary Terminated8.2SPCU.FMS guardin PrimaryTerminated fromSPCU.Primary_SU to SPCU.FMSFMS.SPCU PrimaryisTerminated8.2CPUfail from SPCU.Primary_PU toSPCU.Primary_SUSU.SPCU_Primary Terminated8.4SPCU.FMS guardin PrimaryTerminated fromSPCU.Primary_SU to SPCU.FMSFMS.SPCU PrimaryisTerminated9.1 SPCU.Backup_PU Failure PU.SPCU TerminatedCPUfail from SPCU.Backup_PU toSPCU.Backup_SUSU.SPCU_Backup Terminated9.2SPCU.FMS guardin BackupTerminated fromSPCU.Backup_SU to SPCU.FMSFMS.SPCU Down9.3CPUfail from SPCU.Backup_PU toSPCU.Backup_SUSU.SPCU_Backup Terminated9.4SPCU.FMS guardin BackupTerminated fromSPCU.Backup_SU to SPCU.FMSFMS.SPCU Down21

Tool Set Capabilities for Quantitative EvaluationAADL Architecture and Error ModelsMobius Stochastic AnalysisNetwork ModelResults22

Conclusions• A new generation tool set for quantitative stochastic analysis andqualitative Failure Modes and Effects Analysis (FMEAs) for spacesystems is under development– Based on use of the Architecture Analysis and Design Language (AADL)– Graphically oriented– Modularized with reusable components• Automated Generation of FMEA/CA enables multiple iterationsanalyses throughout all stages of the design– Allows design alternatives to be evaluated• Strategies for recovering from computing disruptions• Handling failure propagation and common mode failures– Enables safety and reliability problems to be identified early• Of critical importance to all users and stakeholders• Significant economic value where products liability is an issue becauseof conforming and exceeding standard of care23

AcronymsADAPT: AADL Architectural models to stochastic Petri nets through model Transformation,AADL: Architecture Analysis & Design LanguageFMEA: Failure Mode and Effects AnalysisFMEA/CA: FMEA /Criticality AnalysisOSATE: Open Source AADL Tool Environment (Software tool integrated into Eclipse)SAE: Society of Automotive EngineersSAN: Stochastic Analysis NetworkTOPCASED: Toolkit In OPen source for Critical Applications & SystEms Development24

References• A. Rugina, K. Kanoun, M Kaaniche, “The ADAPT Tool: From AADL Architectural Models to Stochastic Petri Netsthrough Model Transformation,” 7th European Dependable Computing Conference (EDCC), Kaunas : Lituanie (2008)• Peter Feiler and Anna Rugina, Dependability Modeling with the Architecture Analysis & Design Language (AADL),Software Engineering Institute report CMU/SEI-2007-TN-043, July 2007, available from www.sei.cmu.edu• D. D. Deavours, G. Clark, T. Courtney, D. Daly, S. Derisavi, J. M. Doyle, W. H. Sanders, and P. G. Webster, “TheMobius framework and its implementation,” IEEE Trans. on Soft. Eng., vol. 28, no. 10, pp. 956–969, October 2002.• IEC 60812 (1985) Analysis techniques for system reliability - Procedures for failure mode and effect analysis (FMEA) ,International Electrotechnical Commission,• SAE AS5506 (2004), Aerospace Standard: Architecture Analysis and Design Language available online fromwww.sae.org• SAE ARP 5580 (2001) Recommended Failure Modes and Effects Analysis (FMEA) Practices for Non-AutomobileApplications, Society of Automotive Engineers, available online from www.sae.org• SAE J1739 (2002) Potential Failure Mode and Effects Analysis in Design (Design FMEA) and Potential Failure Modeand Effects Analysis in Manufacturing and Assembly Processes (Process FMEA) and Effects Analysis for Machinery(Machinery FMEA) , Society of Automotive Engineers, available online from www.sae.org• SEMATECH (1992) “Failure Modes and Effects Analysis (FMEA): A Guide for Continuous Improvement for theSemiconductor Equipment Industry”, Technology Transfer #92020963B-ENG, available online athttp://www.ismi.sematech.org/docubase/abstracts/92020963B-ENG.htmAll trademarks, service marks, and trade names are the property of their respectiveowners25