modeling and analysis of fault-tolerant systems using the compass ...

MODELING AND ANALYSIS OFFAULT-TOLERANT SYSTEMS USINGTHE COMPASS TOOLSETViet Yen NguyenSoftware Modeling and Verification GroupRWTH Aachen UniversitySystem Software Engineering SectionEuropean Space AgencyWebEx meeting on 16 Sept. 2011joint work with Marco Bozzano, Alessandro Cimatti,Joost-Pieter Katoen, Thomas Noll, Xavier Olive, Marco Roveriand Yuri Yushtein

AGENDABIRTH OF COMPASSMODELINGREQUIREMENTS SPECIFICATIONANALYSIS & TOOLINGCASE STUDIESEPILOGUELIVE DEMONSTRATION

BIRTH OF COMPASS

WHO’S WHO: JOINT INDUSTRIAL-ACADEMIC R&DConsortium• RWTH Aachen UniversitySoftware Modeling andVerification Group• Fondazione Bruno KesslerEmbedded Systems Group• Thales Alenia SpaceWorld-wide #1 in satellite systems• European Space AgencySystem Software Engineering Section2011, Viet Yen Nguyen FOR PUBLIC USE BIRTH OF COMPASS - 4/50

DOMAIN: FAULT-TOLERANT SPACE SYSTEM ARCHITECTURESExoMars Rover• 4 to 21 min. for radio latency to earth.• XXX Martian days autonomous survival.Autonomous Transfer Vehicle• human-rated: multiple-failure tolerant design.• autonomous docking with four overrides:HOLD, RETREAT, ESCAPE, ABORT2011, Viet Yen Nguyen FOR PUBLIC USE BIRTH OF COMPASS - 5/50

OVERVIEW: CURRENT LIMITATIONS VERSUS COMPASS SOLUTIONSLimitationSimuLinkRtUMLUMLHardwareSoftwareSafety/DependabilityRelexStochasticTimedPetri NetSysMLPPTShapesSystem2011, Viet Yen Nguyen FOR PUBLIC USE BIRTH OF COMPASS - 6/50

OVERVIEW: CURRENT LIMITATIONS VERSUS COMPASS SOLUTIONSLimitationSimuLinkRtUMLUMLHW verified independently ofSW with exaggerated mutualassumptions.HardwareSoftwareSafety/DependabilityRelexStochasticTimedPetri NetSysMLPPTShapesSystem2011, Viet Yen Nguyen FOR PUBLIC USE BIRTH OF COMPASS - 6/50

OVERVIEW: CURRENT LIMITATIONS VERSUS COMPASS SOLUTIONSLimitationSimuLinkRtUMLUMLHW verified independently ofSW with exaggerated mutualassumptions.HardwareSoftwareSafety & dependabilityanalyses are isolated fromHW/SW models.Safety/DependabilityRelexStochasticTimedPetri NetSysMLPPTShapesSystem2011, Viet Yen Nguyen FOR PUBLIC USE BIRTH OF COMPASS - 6/50

OVERVIEW: CURRENT LIMITATIONS VERSUS COMPASS SOLUTIONSLimitationHardwareSafety/DependabilitySimuLinkRelexStochasticTimedPetri NetRtUMLUMLSysMLPPTShapesSoftwareSystemHW verified independently ofSW with exaggerated mutualassumptions.Safety & dependabilityanalyses are isolated fromHW/SW models.Multiple modeling formalismsfor different system aspects(e.g. real-time, probabilistic, hybrid).2011, Viet Yen Nguyen FOR PUBLIC USE BIRTH OF COMPASS - 6/50

OVERVIEW: CURRENT LIMITATIONS VERSUS COMPASS SOLUTIONSLimitationHardwareSafety/DependabilitySimuLinkRelexStochasticTimedPetri NetRtUMLUMLSysMLPPTShapesSoftwareSystemHW verified independently ofSW with exaggerated mutualassumptions.Safety & dependabilityanalyses are isolated fromHW/SW models.Multiple modeling formalismsfor different system aspects(e.g. real-time, probabilistic, hybrid).Non-nominal operationalmodes are overly abstracted tofit various models.2011, Viet Yen Nguyen FOR PUBLIC USE BIRTH OF COMPASS - 6/50

OVERVIEW: CURRENT LIMITATIONS VERSUS COMPASS SOLUTIONSSolutionHardwareSafety/DependabilityHardware SoftwareComponents ComponentsError ModelsSoftwareSystemCombine HW, SW and theirbindings + ...error models ... +real-time, probabilistic andhybrid aspects ... +non-nominal modes in a singleintegrated model.2011, Viet Yen Nguyen FOR PUBLIC USE BIRTH OF COMPASS - 7/50

ALL-IN-ONE FOR RAISING THE BAR OF SYSTEM-LEVEL ENGINEERINGSpecificationPatterns forRequirementsTooling & AnalysisModel CheckingSLIM =AADL + Error +Behavior +Formal SemanticsCOMPASSProjectIndustrialCase Studies2011, Viet Yen Nguyen FOR PUBLIC USE BIRTH OF COMPASS - 8/50

MODELING

AADL: INDUSTRY STANDARD FOR MODELING EMBEDDED SYSTEMSParadigm• 1989 MetaH• 1998 SAE AS-2C• Architecture-based andmodel-driven top-down andbottom-up engineering• Real-time and performancecritical distributed systems• Complements component-basedproduct-line development• 2004 AADL 1.0• 2006 Error Annex• 2009 AADL 2.02011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 10/50

SLIM: AADL-BASED MODELING LANGUAGE FOR FORMAL ANALYSISSLIM = AADL+ Behavior+ Error Annex+ Formal Semantics+ ɛSystem-Level Integrated Modeling:• Covers major part of AADL 1.0.• Functional, real-time and hybrid behavior.• Poisson-distributed errors.• Mathematical characterization of behavior.• Minor adaptations needed for formalization.2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 11/50

RUNNING SLIM EXAMPLE: REDUNDANT POWER SYSTEMPoweremptybatt1primarybackupemptybatt2We shall show:• hybrid behaviour of the batteries,• composition of the power system,• semantics as transition systems,• interweaving of errors.voltagevoltagevoltage2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 12/50

SLIM: MODEL OF A SINGLE BATTERYCOMPONENT TYPE AND IMPLEMENTATIONdevice type Batteryend Battery;device implementation Battery.Impend Battery.Imp;2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 13/50

SLIM: MODEL OF A SINGLE BATTERYCOMPONENT TYPE DEFINES THE INTERFACEdevice type Batteryfeaturesempty: out event port;voltage: out data port real initially 6.0;end Battery;device implementation Battery.Impend Battery.Imp;2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 13/50

SLIM: MODEL OF A SINGLE BATTERYADDING MODES BEHAVIOURdevice type Batteryfeaturesempty: out event port;voltage: out data port real initially 6.0;end Battery;device implementation Battery.Impmodescharged: activation modedepleted: modetransitionscharged -[]-> charged;charged -[empty]-> depleted;depleted -[]-> depleted;end Battery.Imp;2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 13/50

SLIM: MODEL OF A SINGLE BATTERYADDING HYBRID BEHAVIOURdevice type Batteryfeaturesempty: out event port;voltage: out data port real initially 6.0;end Battery;device implementation Battery.Impsubcomponentsenergy: data continuous initially 100.0;modescharged: activation modewhile energy’=-0.02 and energy>=20.0;depleted: modewhile energy’=-0.03;transitionscharged -[then voltage:=energy/50.0+4.0]-> charged;charged -[empty when energy depleted;depleted -[then voltage:=energy/50.0+4.0]-> depleted;end Battery.Imp;2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 13/50

SLIM: MODEL OF OVERALL POWER SYSTEMPOWER SYSTEM WITH BATTERY SUBCOMPONENTSsystem Powerfeaturesvoltage: out data port real;end Power;system implementation Power.Impsubcomponentsbatt1: device Battery.Impbatt2: device Battery.Impend Power.Imp;2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 14/50

SLIM: MODEL OF OVERALL POWER SYSTEMADDING DYNAMIC RECONFIGURATIONsystem Powerfeaturesvoltage: out data port real;end Power;system implementation Power.Impsubcomponentsbatt1: device Battery.Imp in modes (primary);batt2: device Battery.Imp in modes (backup);modesprimary: initial mode;backup: mode;transitionsprimary -[batt1.empty]-> backup;backup -[batt2.empty]-> primary;end Power.Imp;2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 14/50

SLIM: MODEL OF OVERALL POWER SYSTEMADDING PORT CONNECTIONSsystem Powerfeaturesvoltage: out data port real;end Power;system implementation Power.Impsubcomponentsbatt1: device Battery.Imp in modes (primary);batt2: device Battery.Imp in modes (backup);connectionsdata port batt1.voltage -> voltage in modes (primary);data port batt2.voltage -> voltage in modes (backup);modesprimary: initial mode;backup: mode;transitionsprimary -[batt1.empty]-> backup;backup -[batt2.empty]-> primary;end Power.Imp;2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 14/50

SLIM: FORMAL SEMANTICS AS NETWORK OF EVENT-DATA AUTOMATANetwork ofEvent-DataAutomata ≈ communicating state machines+ linear differential equations on states+ clocks and real-timed transitions+ Markovian transitions (probabilistic)Formalization by mapping:• 1 SLIM component Event-Data Automaton• SLIM hierarchy Network of Event-Data Automaton2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 15/50

SLIM: TRANSITIONS BY A NETWORK OF EVENT-DATA AUTOMATA• States := (M 1 × V 1 ) × . . . × (M n × V n )• Transitions determined by active EDAs:1. Perform local transitions:• timed local transition in all EDAs or• internal transition in EDA or• multiway event communication from EDA to ≥ 1 connected EDAs2. Initialize (re-)activated subcomponents3. Establish consistency w.r.t. DC (copy source → target data port)2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 16/50

SLIM: TRANSITIONS BY A NETWORK OF EVENT-DATA AUTOMATA• States := (M 1 × V 1 ) × . . . × (M n × V n )• Transitions determined by active EDAs:1. Perform local transitions:• timed local transition in all EDAs or• internal transition in EDA or• multiway event communication from EDA to ≥ 1 connected EDAs2. Initialize (re-)activated subcomponents3. Establish consistency w.r.t. DC (copy source → target data port)Example (Power system)〈m=primary, v=6.0〉 〈m=charged, e=100.0, v=6.0〉 〈m=charged, e=100.0, v=6.0〉2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 16/50

SLIM: TRANSITIONS BY A NETWORK OF EVENT-DATA AUTOMATA• States := (M 1 × V 1 ) × . . . × (M n × V n )• Transitions determined by active EDAs:1. Perform local transitions:• timed local transition in all EDAs or• internal transition in EDA or• multiway event communication from EDA to ≥ 1 connected EDAs2. Initialize (re-)activated subcomponents3. Establish consistency w.r.t. DC (copy source → target data port)Example (Power system)〈m=primary, v=6.0〉 〈m=charged, e=100.0, v=6.0〉 〈m=charged, e=100.0, v=6.0〉⇓ 40.0〈m=primary, v=6.0〉 〈m=charged, e=20.0, v=6.0〉 〈m=charged, e=100.0, v=6.0〉2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 16/50

SLIM: TRANSITIONS BY A NETWORK OF EVENT-DATA AUTOMATA• States := (M 1 × V 1 ) × . . . × (M n × V n )• Transitions determined by active EDAs:1. Perform local transitions:• timed local transition in all EDAs or• internal transition in EDA or• multiway event communication from EDA to ≥ 1 connected EDAs2. Initialize (re-)activated subcomponents3. Establish consistency w.r.t. DC (copy source → target data port)Example (Power system)〈m=primary, v=6.0〉 〈m=charged, e=100.0, v=6.0〉 〈m=charged, e=100.0, v=6.0〉⇓ 40.0〈m=primary, v=6.0〉 〈m=charged, e=20.0, v=6.0〉 〈m=charged, e=100.0, v=6.0〉⇓ τ〈voltage:=...〉〈m=primary, v=4.4〉 〈m=charged, e=20.0, v=4.4〉 〈m=charged, e=100.0, v=6.0〉2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 16/50

SLIM: TRANSITIONS BY A NETWORK OF EVENT-DATA AUTOMATA• States := (M 1 × V 1 ) × . . . × (M n × V n )• Transitions determined by active EDAs:1. Perform local transitions:• timed local transition in all EDAs or• internal transition in EDA or• multiway event communication from EDA to ≥ 1 connected EDAs2. Initialize (re-)activated subcomponents3. Establish consistency w.r.t. DC (copy source → target data port)Example (Power system)〈m=primary, v=6.0〉 〈m=charged, e=100.0, v=6.0〉 〈m=charged, e=100.0, v=6.0〉⇓ 40.0〈m=primary, v=6.0〉 〈m=charged, e=20.0, v=6.0〉 〈m=charged, e=100.0, v=6.0〉⇓ τ〈voltage:=...〉〈m=primary, v=4.4〉 〈m=charged, e=20.0, v=4.4〉 〈m=charged, e=100.0, v=6.0〉⇓ τ〈empty〉〈m=backup, v=6.0〉 〈m=depleted, e=20.0, v=4.4〉 〈m=charged, e=100.0, v=6.0〉2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 16/50

SLIM: TRANSITIONS BY A NETWORK OF EVENT-DATA AUTOMATA• States := (M 1 × V 1 ) × . . . × (M n × V n )• Transitions determined by active EDAs:1. Perform local transitions:• timed local transition in all EDAs or• internal transition in EDA or• multiway event communication from EDA to ≥ 1 connected EDAs2. Initialize (re-)activated subcomponents3. Establish consistency w.r.t. DC (copy source → target data port)Example (Power system)〈m=primary, v=6.0〉 〈m=charged, e=100.0, v=6.0〉 〈m=charged, e=100.0, v=6.0〉⇓ 40.0〈m=primary, v=6.0〉 〈m=charged, e=20.0, v=6.0〉 〈m=charged, e=100.0, v=6.0〉⇓ τ〈voltage:=...〉〈m=primary, v=4.4〉 〈m=charged, e=20.0, v=4.4〉 〈m=charged, e=100.0, v=6.0〉⇓ τ〈empty〉〈m=backup, v=6.0〉 〈m=depleted, e=20.0, v=4.4〉 〈m=charged, e=100.0, v=6.0〉⇓ 40.0〈m=backup, v=6.0〉 〈m=depleted, e=20.0, v=4.4〉 〈m=charged, e=20.0, v=6.0〉2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 16/50

SLIM: TRANSITIONS BY A NETWORK OF EVENT-DATA AUTOMATA• States := (M 1 × V 1 ) × . . . × (M n × V n )• Transitions determined by active EDAs:1. Perform local transitions:• timed local transition in all EDAs or• internal transition in EDA or• multiway event communication from EDA to ≥ 1 connected EDAs2. Initialize (re-)activated subcomponents3. Establish consistency w.r.t. DC (copy source → target data port)Example (Power system)〈m=primary, v=6.0〉 〈m=charged, e=100.0, v=6.0〉 〈m=charged, e=100.0, v=6.0〉⇓ 40.0〈m=primary, v=6.0〉 〈m=charged, e=20.0, v=6.0〉 〈m=charged, e=100.0, v=6.0〉⇓ τ〈voltage:=...〉〈m=primary, v=4.4〉 〈m=charged, e=20.0, v=4.4〉 〈m=charged, e=100.0, v=6.0〉⇓ τ〈empty〉〈m=backup, v=6.0〉 〈m=depleted, e=20.0, v=4.4〉 〈m=charged, e=100.0, v=6.0〉⇓ 40.0〈m=backup, v=6.0〉 〈m=depleted, e=20.0, v=4.4〉 〈m=charged, e=20.0, v=6.0〉⇓ · · ·2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 16/50

SLIM: INCLUDING ERRONEOUS TO NOMINAL BEHAVIORNominal Model =SLIM componentsError ModelsFault InjectionsAutomaticModel ExtensionExtended Model =nominal +error effects +degraded behavior2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 17/50

SLIM: ERROR MODELING AND FAULT INJECTIONerror model BatteryFailurefeaturesok: initial state;dead: error state;batteryDied: out error propagation;end BatteryFailure;error model implementation BatteryFailure.Impeventsfault: error event occurrence poisson 0.01;transitionsok -[fault]-> dead;dead -[batteryDied]-> dead;end BatteryFailure.Imp;2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 18/50

SLIM: ERROR MODELING AND FAULT INJECTIONerror model BatteryFailurefeaturesok: initial state;dead: error state;batteryDied: out error propagation;end BatteryFailure;error model implementation BatteryFailure.Impeventsfault: error event occurrence poisson 0.01;transitionsok -[fault]-> dead;dead -[batteryDied]-> dead;end BatteryFailure.Imp;Fault InjectionIn error state dead, voltage:=02011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 18/50

SLIM: BATTERY COMPONENTNOMINAL SPECIFICATIONdevice type Batteryfeaturesempty: out event port;voltage: out data port real initially 6.0;end Battery;device implementation Battery.Impsubcomponentsenergy: data continuous initially 100.0;modescharged: activation mode while ...;depleted: mode while ...;transitionscharged -[then voltage:=...]-> charged;charged -[empty when energy depleted;depleted -[then voltage:=...]-> depleted;end Battery.Imp;2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 19/50

SLIM: BATTERY COMPONENT AFTER MODEL EXTENSIONPRODUCT CONSTRUCTION FOR MODES AND ERROR STATESdevice type Batteryfeaturesempty: out event port;voltage: out data port real initially 6.0;end Battery;device implementation Battery.Impsubcomponentsenergy: data continuous initially 100.0;modescharged#ok: activation mode while ...;depleted#ok, charged#dead, depleted#dead: mode while ...;transitionscharged -[then voltage:=...]-> charged;charged -[empty when energy depleted;depleted -[then voltage:=...]-> depleted;end Battery.Imp;2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 19/50

SLIM: BATTERY COMPONENT AFTER MODEL EXTENSIONINTEGRATE NOMINAL TRANSITIONSdevice type Batteryfeaturesempty: out event port;voltage: out data port real initially 6.0;end Battery;device implementation Battery.Impsubcomponentsenergy: data continuous initially 100.0;modescharged#ok: activation mode while ...;depleted#ok, charged#dead, depleted#dead: mode while ...;transitionscharged#ok -[then voltage:=...]-> charged#ok;charged#ok -[empty when energy depleted#ok;depleted#ok -[then voltage:=...]-> depleted#ok;end Battery.Imp;2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 19/50

SLIM: BATTERY COMPONENT AFTER MODEL EXTENSIONADD FAULT INJECTIONSdevice type Batteryfeaturesempty: out event port;voltage: out data port real initially 6.0;end Battery;device implementation Battery.Impsubcomponentsenergy: data continuous initially 100.0;modescharged#ok: activation mode while ...;depleted#ok, charged#dead, depleted#dead: mode while ...;transitionscharged#ok -[then voltage:=...]-> charged#ok;charged#ok -[empty when energy depleted#ok;depleted#ok -[then voltage:=...]-> depleted#ok;charged#ok -[then voltage:=0]-> charged#dead;depleted#ok -[then voltage:=0]-> depleted#dead;end Battery.Imp;2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 19/50

SLIM: BATTERY COMPONENT AFTER MODEL EXTENSIONNOMINAL TRANSITIONS WITH FAULT EFFECTSdevice type Batteryfeaturesempty: out event port;voltage: out data port real initially 6.0;end Battery;device implementation Battery.Impsubcomponentsenergy: data continuous initially 100.0;modescharged#ok: activation mode while ...;depleted#ok, charged#dead, depleted#dead: mode while ...;transitionscharged#ok -[then voltage:=...]-> charged#ok;charged#ok -[empty when energy depleted#ok;depleted#ok -[then voltage:=...]-> depleted#ok;charged#ok -[then voltage:=0]-> charged#dead;depleted#ok -[then voltage:=0]-> depleted#dead;charged#dead -[then voltage:=0]-> charged#dead;charged#dead -[empty when energy depleted#dead;depleted#dead -[then voltage:=0]-> depleted#dead;end Battery.Imp;2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 19/50

SLIM: BATTERY COMPONENT AFTER MODEL EXTENSIONADD ERROR PROPAGATIONSdevice type Batteryfeaturesempty: out event port;voltage: out data port real initially 6.0;batteryDied: out event port;end Battery;device implementation Battery.Impsubcomponentsenergy: data continuous initially 100.0;modescharged#ok: activation mode while ...;depleted#ok, charged#dead, depleted#dead: mode while ...;transitionscharged#ok -[then voltage:=...]-> charged#ok;charged#ok -[empty when energy depleted#ok;depleted#ok -[then voltage:=...]-> depleted#ok;charged#ok -[then voltage:=0]-> charged#dead;depleted#ok -[then voltage:=0]-> depleted#dead;charged#dead -[then voltage:=0]-> charged#dead;charged#dead -[empty when energy depleted#dead;depleted#dead -[then voltage:=0]-> depleted#dead;depleted#dead -[batteryDied]-> depleted#dead;charged#dead -[batteryDied]-> charged#dead;end Battery.Imp;2011, Viet Yen Nguyen FOR PUBLIC USE MODELING - 19/50

REQUIREMENTSSPECIFICATION

PATTERNS: LOGIC-FREE SPECIFICATION OF REQUIREMENTSPatterns• The system shall have a behavior where with probability higherthan 0.98 it is the case that voltage ≥ 80 holds continouslywithin time bound [0,10] .• The system shall have a behavior where x ≤ voltage ≤ yglobally holds.Logic|(by automatic transformation)↓• P >0.98 [□ [0,10] (voltage ≥ 80)]• □(x ≤ voltage ≤ y)(Continuous Stochastic Logic)(Linear Temporal Logic)2011, Viet Yen Nguyen FOR PUBLIC USE REQUIREMENTS SPECIFICATION - 21/50

ANALYSIS & TOOLING

COMPASS TOOLSET 2.2 FEATURES WITH RELATION TO INPUTSInputIntermediateArtefactSystem/HW/SWSafety/DependabilityAnalyses/OutputNominalModelError ModelFaultInjectionsReq's(patterns)ExtendedModelFormal Canonical ModelsState Space(LTS/SMT)NuSMVMarkovChainReq's(logic)RATCorrectness- Model checking(hybrid/discrete)- Simulation- Deadlock checkingFDIR Effectiveness- Fault detection- Fault isolation- Fault recovery- DiagnosabilitySafety/Depend.- Dynamic FTA- Dynamic FMEA- Fault TolerancePerformability- Performance eval.- Probabilistic riskassessment offault tree.Validation- Consistency check- Assertion check- Simulation2011, Viet Yen Nguyen FOR PUBLIC USE ANALYSIS & TOOLING - 23/50

VALIDATION: ARE WE BUILDING THE RIGHT SYSTEM?Find specificationinconsistencies inthe RequirementsSpecification beforedetailed designstarts.Analyses• Property consistency: do the requirements contradict eachother?• Property assurance: are the requirements too strict/permissive?2011, Viet Yen Nguyen FOR PUBLIC USE ANALYSIS & TOOLING - 24/50

CORRECTNESS: ARE WE BUILDING THE SYSTEM RIGHT?Verification of theSLIM model (withoptionally failures)againstrequirements.Analyses• Model simulation• Model checking (also known as Exhaustive Testing)• Deadlock checking2011, Viet Yen Nguyen FOR PUBLIC USE ANALYSIS & TOOLING - 25/50

SAFETY & DEPENDABILITY: MAPPING FAILURE MODES ↔ FAULTS?Generation of safety& dependabilityartefacts from theSLIM model.Analyses• (Probabilistic) dynamic Fault Tree Analysis (FTA)• (Dynamic) Failure modes and effects analysis (FMEA)• Fault tolerance evaluation2011, Viet Yen Nguyen FOR PUBLIC USE ANALYSIS & TOOLING - 26/50

SAFETY & DEPENDABILITY: MAPPING FAILURE MODES ↔ FAULTS?Generation of safety& dependabilityartefacts from theSLIM model.Analyses• (Probabilistic) dynamic Fault Tree Analysis (FTA)• (Dynamic) Failure modes and effects analysis (FMEA)• Fault tolerance evaluation2011, Viet Yen Nguyen FOR PUBLIC USE ANALYSIS & TOOLING - 26/50

FDIR: WILL THE SYSTEM RECOVER FROM A FAULT?Analysis of the FDIRpart in the SLIMmodel containing theobservable tags.Analyses• Fault detection analysis: how does the system detect faults?• Fault isolation analysis: can the system distinguish faults?• Fault recovery analysis: does the system recover after a fault?• Diagnosability: are there sufficient sensors to detect faults?2011, Viet Yen Nguyen FOR PUBLIC USE ANALYSIS & TOOLING - 27/50

FDIR: WILL THE SYSTEM RECOVER FROM A FAULT?Analysis of the FDIRpart in the SLIMmodel containing theobservable tags.Analyses• Fault detection analysis: how does the system detect faults?• Fault isolation analysis: can the system distinguish faults?• Fault recovery analysis: does the system recover after a fault?• Diagnosability: are there sufficient sensors to detect faults?2011, Viet Yen Nguyen FOR PUBLIC USE ANALYSIS & TOOLING - 27/50

FDIR: WILL THE SYSTEM RECOVER FROM A FAULT?Analysis of the FDIRpart in the SLIMmodel containing theobservable tags.Analyses• Fault detection analysis: how does the system detect faults?• Fault isolation analysis: can the system distinguish faults?• Fault recovery analysis: does the system recover after a fault?• Diagnosability: are there sufficient sensors to detect faults?2011, Viet Yen Nguyen FOR PUBLIC USE ANALYSIS & TOOLING - 27/50

FDIR: WILL THE SYSTEM RECOVER FROM A FAULT?Analysis of the FDIRpart in the SLIMmodel containing theobservable tags.Analyses• Fault detection analysis: how does the system detect faults?• Fault isolation analysis: can the system distinguish faults?• Fault recovery analysis: does the system recover after a fault?• Diagnosability: are there sufficient sensors to detect faults?2011, Viet Yen Nguyen FOR PUBLIC USE ANALYSIS & TOOLING - 27/50

PERFORMABILITY: CHANCE & TIME TO FAILURE IN DEGRADED STATE?Markovianperformanceevaluation of theSLIM model foravailability andsensitivity.Analyses• Performability: how long and with which probability can thesystem sustain degraded operations?2011, Viet Yen Nguyen FOR PUBLIC USE ANALYSIS & TOOLING - 28/50

CASE STUDIES

CASE 1: SATELLITE REGULATION SYSTEMChallenges:• Hardware (sensors, heaters) and software (control) co-engineering• Hybrid behavior (temperatures)• Dynamic reconfiguration (redundancy)• State-space explosion2011, Viet Yen Nguyen FOR PUBLIC USE CASE STUDIES - 30/50

CASE 2: SATELLITE FDIR SYSTEMGoalAssess effectiveness of FDIR measuresModel components:• satellite mode management during transfer-to-orbit phase• AOCS (Attitude and Orbit Control System) mode management• abstraction of AOCS equipment (sensors, gyroscope, ...)• FDIR action sequenceAnalysis problems:• identification of failures leading to a given FDIR level• identification of failures entailing a system reconfiguration• impact of reconfiguration on satellite and AOCS mode2011, Viet Yen Nguyen FOR PUBLIC USE CASE STUDIES - 31/50

CASE 3: SATELLITE PLATFORMSatellitePayloadPlatformPayload is mission-specificequipment, e.g.:• telecom transponders,• navigation signals,• earth observation telemetry(weather, radiation, salinity).Platform keeps the satelliteorbiting in space, consists of:• attitude & orbital control,• power distribution,• data handling,• communications,• thermal regulation,• propulsion.2011, Viet Yen Nguyen FOR PUBLIC USE CASE STUDIES - 32/50

CASE 3: SATELLITE PLATFORMVerification & Validation Objectives• Ensure nominal and degraded conditions are handled correctlyby the fault management system.• Ensure performance and risks are within specified limits.Model Characteristicš Functionaľ Probabilistič Continuous Real-Timě HybridComponents: 99Modes: 217Faults: 21Recoveries: 9State Space of Nominal Behaviour: 48,421,100 statesRequirement Metrics• Functional properties: 32• Probabilistic: 22011, Viet Yen Nguyen FOR PUBLIC USE CASE STUDIES - 33/50

CASE 3: SATELLITE PLATFORMSetup: Intel Xeon 2.33 GHz machine with 16 GB RAM.Fault injections: earth sensor failure.AnalysisTime (min:sec)Deadlock checking 3:41Model checking “recovers from sensor failure” 10:34Fault tree analysis “sensor reading incorrect” 6:35Fault tree evaluation “sensor reading incorrect” 0:02Fault tolerance evaluation 0:06Dynamic fault tree analysis “sensor reading incorrect” 10:51Dynamic fault tree evaluation “sensor reading incorrect” 0:03Fault detection “sensor failed” 22:47Fault isolation “sensor failed” 4:59Fault recovery “sensor failed” 10:24FMEA “sensor failed” 18:10Performability > 552:00 1Diagnosability > 3823:00 21 ran out of memory2 aborted after 63 hours2011, Viet Yen Nguyen FOR PUBLIC USE CASE STUDIES - 34/50

GENERAL: MORE COMPLEX MODELS MEANS MORE TIME TO ANALYZE30000Verification Time22500Seconds15000750002 3 4 5 6 7 8 9 10Degree of RedundancyNuSMV SigRef MRMC2011, Viet Yen Nguyen FOR PUBLIC USE CASE STUDIES - 35/50

GENERAL: EVALUATION OUTCOME+ Abstraction level of models is appropriate• mode transition systems• not source code+ Support of incremental approach to system design• start with abstract functional representation• refinement without breaking structure of model• separation of component interface and implementation+ Analyses give valuable feedback to system designer− Missing (automatic) link between AADL and engineering models(UML, Matlab Simulink)• integration into engineering tool suite− Toolset performance on timed and hybrid systems need furtherstudy2011, Viet Yen Nguyen FOR PUBLIC USE CASE STUDIES - 36/50

EPILOGUE

ALL-IN-ONE FOR RAISING THE BAR OF SYSTEM-LEVEL ENGINEERINGSpecificationPatterns forRequirementsTooling & AnalysisModel CheckingSLIM =AADL + Error +Behavior +Formal SemanticsCOMPASSProjectIndustrialCase Studies2011, Viet Yen Nguyen FOR PUBLIC USE EPILOGUE - 38/50

COMPASS PROJECT: 2008-20111. Project Kick-Off February 20082. SLIM Language Design3. Software Tool Specification4. Software Design Document5. Formal SLIM Semantics October 20086. Prototype Tool Implementation April 20097. Prototype Evaluation8. Final Tool Implementation December 20099. Final Tool Evaluation March 201010. CCN Kick-Off October 201011. CCN End March 20112011, Viet Yen Nguyen FOR PUBLIC USE EPILOGUE - 39/50

POST-COMPASS: RESEARCH & TECHNOLOGY DEVELOPMENT• SLIM slicing March 2010Smaller models by semantics-preservationreduction using requirements as driver.• Compositional model checking September 2010Faster analysis of models by exploiting thehierarchical and component-oriented structure.• SLIM graphical modeler January 2011User-friendly and graphical drag-&-dropconstruction of SLIM models.• Impact analysis April 2011Understand scope of cause-&-effect of(hardware/software) functions for classificationof criticalities/priorities.• Contribution to AADL standard October 2011Leverage COMPASS-experience of buildingtheoretically solid analyses & tools.2011, Viet Yen Nguyen FOR PUBLIC USE EPILOGUE - 40/50

AVAILABILITY & FUTURE INFORMATIONCOMPASS Toolset binary & source availability is restricted to legalpersons of ESA-member states. For non-ESA member states, a technologytransfer procedure needs to be initiated. Contact me for details.• Management summary(Katoen & Noll, 11 th Public Service Review: EU Science & Tech. 2011)• Technical overview(Yushtein et. al, IEEE Space Mission Challenges 2011)• Technical realization(Bozzano et. al, Oxford Computer Journal 2011)• SLIM formal semantics(Bozzano et. al, IEEE MEMOCODE 2009)• SLIM model checker(Bozzano et. al, CAV 2010)• Website(http://compass.informatik.rwth-aachen.de/)2011, Viet Yen Nguyen FOR PUBLIC USE EPILOGUE - 41/50

ROUND TABLE DISCUSSION POINTS• What kind of models do you want to capture?What characteristics? (probabilistic, real-timed, . . . )• Kickstarting a formal modeling & analysis pilot?• How does feature/analysis . . . work?• How does COMPASS fit in the traditionalengineering/development process?• . . . ?2011, Viet Yen Nguyen FOR PUBLIC USE EPILOGUE - 42/50

LIVE DEMONSTRATION

REDUNDANT SENSOR-FILTER ACQUISITION SYSTEM WITH FDIRAcquisitionalarmSalarmSMonitoralarmFalarmFswitchSswitchFswitchswitchvalueSensorsFilterssensor1filter1sensor2filter22011, Viet Yen Nguyen FOR PUBLIC USE LIVE DEMONSTRATION - 44/50

MODELING THE SENSOR-BANK IN SLIMsystem Sensorsfeaturesoutput: out data port int default 1;switch: in event port;end Sensors;system implementation Sensors.Implsubcomponentssensor1: device Sensor in modes (Primary);sensor2: device Sensor in modes (Backup);connectionsdata port sensor1.output -> output in modes (Primary);data port sensor2.output -> output in modes (Backup);modesPrimary: activation mode; Backup: mode;transitionsPrimary -[switch]-> Backup;end Sensors.Impl;switchSensorssensor1sensor22011, Viet Yen Nguyen FOR PUBLIC USE LIVE DEMONSTRATION - 45/50

MODELING A SENSOR IN SLIMdevice Sensorfeaturesoutput: out data port int default 1;end Sensor;sensor1device implementation Sensor.ImplmodesCycle: activation mode;transitionsCycle -[when output < 5 then output := output + 1]-> Cycle;end Sensor.Impl;2011, Viet Yen Nguyen FOR PUBLIC USE LIVE DEMONSTRATION - 46/50

ERROR MODELS OF SENSOR & FILTER FAILURESSensorFailuresFilterFailures0.051Degrade0.007OK0.00015DeadOK0.0510.007Dead0.083Drifted0.000152011, Viet Yen Nguyen FOR PUBLIC USE LIVE DEMONSTRATION - 47/50

ERROR MODEL OF SENSOR IN SLIMerror model SensorFailuresfeaturesOK: initial state;Drifted: error state;Dead: error state;end SensorFailures;OKSensorFailures0.000150.083Deaderror model implementation SensorFailures.Impleventsdrift: error event occurrence poisson 0.083;die: error event occurrence poisson 0.00001;dieByDrift: error event occurrence poisson 0.00015;transitionsOK -[ die ]-> Dead;OK -[ drift ]-> Drifted;Drifted -[ dieByDrift ]-> Dead;end SensorFailures.Impl;Drifted0.000152011, Viet Yen Nguyen FOR PUBLIC USE LIVE DEMONSTRATION - 48/50

EXTENDED MODEL OF DATA-ACQUISITION SYSTEMAcquisitionalarmSalarmSMonitoralarmFalarmFswitchSswitchFswitchswitchvalueSensorsFiltersDead: output := 15sensor1filter1Dead: output := 0Dead: output := 15sensor2filter2Dead: output := 02011, Viet Yen Nguyen FOR PUBLIC USE LIVE DEMONSTRATION - 49/50

PROPERTIES OF INTEREST• Are the alarms raised when the filters fail?• Which errors lead to a sensor failure?• What are the system effects upon a sensor failure?• Probability that either sensor or the filters die within 76h?• Probability that sensors die before the filters die within 512h?• Probability that the first sensor’s output ≥ 15 within 240h?• Which observables are raised upon a filter failure?• Are the sensor observables sufficient for isolating a failure?• Can we diagnose a sensor failure from the overall system?2011, Viet Yen Nguyen FOR PUBLIC USE LIVE DEMONSTRATION - 50/50