Agile Software Development with Verification and ... - Rally Software
Agile Software Development with Verification and ... - Rally Software
Agile Software Development with Verification and ... - Rally Software
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
<strong>Agile</strong> <strong>Software</strong> <strong>Development</strong> <strong>with</strong><strong>Verification</strong> <strong>and</strong> Validation inHigh Assurance <strong>and</strong>Regulated Environments© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.
Stump Us <strong>with</strong> Your Toughest Questions Q&A Panel of <strong>Agile</strong>,industry <strong>and</strong> <strong>Rally</strong> productexperts st<strong>and</strong>ing by toanswer your questions Enter your question intoto the Q&A window Webinar tech supportavailable here© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.2
Dean Leffingwell<strong>Agile</strong> <strong>Software</strong>RequirementsLean Requirements Practices forTeams, Programs, <strong>and</strong> the EnterpriseDean LeffingwellForeword by Don Reinertsen<strong>Agile</strong> <strong>Software</strong> <strong>Development</strong> SeriesAlistair Cockburn <strong>and</strong> Jim Highsmith,Series Editors© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.3
Michael Meissner, PhDEngineering Executive roles:• Omnyx LLC• Vital Images• Viatronix<strong>Agile</strong> Evangelist• Scrum in company built ground-up• Scrum in established company• Informal agile developmentBeliever:• <strong>Development</strong> needs to be fun• Customer needs to be central• Results matter & introspection is keyBackground• 15+ years experience in <strong>Software</strong><strong>Development</strong> in RegulatedEnvironments Craig LangenfeldRoles:• <strong>Software</strong> Engineer• Project Manager (PMP)• Scrum Master (CSM)• Technical Account Manager/<strong>Agile</strong>Coach<strong>Agile</strong> Evangelist• Implemented <strong>Agile</strong> <strong>and</strong> <strong>Rally</strong> in anumber of enterprise healthcareorganizationsBackground• 5 years of Scaling <strong>Software</strong> Agility at<strong>Rally</strong> <strong>Software</strong>© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.4
Agenda➵ <strong>Agile</strong> Crosses the Chasm to High Assurance➵ An <strong>Agile</strong>, High Assurance Lifecycle Framework➵ Iterating <strong>and</strong> Continuous <strong>Verification</strong>➵ Validating System Increments➵ Impact on Quality Management Systems© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.5
AGILE CROSSES THE CHASMTO HIGH ASSURANCEDEVELOPMENT© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.6
<strong>Agile</strong> gets resultsWe experienced a 20-50%increase in productivity.− BMC Case Studymakes the work moreenjoyable, helps us worktogether, <strong>and</strong> isempowering− MedtronicProductivityQuality<strong>Agile</strong> teams average37-50% faster tomarket− QSM researchMoraleTime toMarket© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.7
<strong>Agile</strong> technical practices drive quality,safety, efficacy…fewer defects were found− Abbott LabsCollectiveOwnershipTest-Driven<strong>Development</strong>CodingSt<strong>and</strong>ardsContinuousIntegrationPairProgrammingQualitySimpleDesignAutomatedTestingRefactoring… of 131 respondents, 88%said quality was better orsignificantly better− Shine Technology SurveyUser StoriesHelps us find bugs earlier− Medtronic© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.8
<strong>Agile</strong> is already in high assurance Abbott Laboratories –– “On the <strong>Agile</strong> project, fewer defects were found…– availability of working software early on was a significant factor.– An agile approach is the approach best suited to development of FDAregulateddevices.” Association for the Advancement of Medical Instrumentation(AAMI)– Technical Information Report (available later this year), using agile forcomplying <strong>with</strong> international st<strong>and</strong>ards <strong>and</strong> FDA AFEI DoD <strong>Agile</strong> <strong>Development</strong> Conference– “<strong>Agile</strong> Methods are in widespread use by the U.S. DoD, Prior … thecommercial industry <strong>and</strong> DoD contractors believed the U.S. DoD wasnot committed to <strong>Agile</strong> , an enormously incorrect assumption…”Sources: Abbott Labs whitepaper: http://www.computer.org/portal/web/csdl/doi/10.1109/AGILE.2009.50.AAMI report: See http://www.aami.org/applications/search/details.cfm?WebID=P1541_D6110DoD Association for Enterprise Information (AFEI): See http://www.afei.org/Pages/default.aspx.(See http://davidfrico.com/afei-2010.doc)© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.9
Surprise? FDA <strong>and</strong> IEC guidance doesNOT recommend waterfall[FDA CDRH 2002] It is important to note, that neither this document, norCFR820.30 itself, constrains development to single pass, stage-gated,waterfall activities. From General Principles of <strong>Software</strong> Validation….. [FDA CDRH 2002]:While this guidance does not recommend any specific life cycle model orany specific technique or method, it does recommend that softwarevalidation <strong>and</strong> verification activities be conducted throughout the lifecycle.From [FDA CDRH 1997] Design Control Guidance for Medical DeviceManufacturers : Although the waterfall model is a useful tool forintroducing design controls, its usefulness in practice is limited… for morecomplex devices, a concurrent engineering model is more representativeof the design processes in use in the industry.IEC62304 medical device st<strong>and</strong>ard states: … these activities <strong>and</strong> tasksmay overlap or interact <strong>and</strong> may be performed iteratively or recursively. Itis not the intent to imply that a waterfall model should be used.© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.10
With respect to this one industry, <strong>and</strong> <strong>with</strong> respect tothese specific guidelines, any notion that we arem<strong>and</strong>ated to apply a single-pass, waterfall model tosoftware development is an industry myth, onewhich has likely been perpetuated by our ownwaterfall past (“we’ve always done it this way”) <strong>and</strong>our existing quality management systems, <strong>and</strong> notbecause “the regulations make us do it”© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.11
AN AGILE, HIGH ASSURANCELIFECYCLE FRAMEWORK© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.12
A Scaled <strong>Agile</strong> Framework ©2011 Leffingwell, LLC.H H H H See <strong>Agile</strong> So)ware Requirements: Lean Requirements Prac8ces for Teams, Programs, <strong>and</strong> the Enterprise <strong>and</strong> www.scalingso/wareagility.wordpress.com
But high assurance development hasadditional requirementsMedical device exemplar: US FDA m<strong>and</strong>ates<strong>Software</strong> <strong>Verification</strong> <strong>and</strong> ValidationUserNeedsReviewDesignInputDesignProcess<strong>Verification</strong>DesignOutputMedicaldeviceValidationSource: FDA CDRH 1997 Design ControlGuidance for Medical Device Manufacturers© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.
So we have additional m<strong>and</strong>atesCode of Federal Regulations CFR 21 Part 830, Subpart C Design Controlsm<strong>and</strong>ates device design verification <strong>and</strong> validation.<strong>Verification</strong>Provides objective evidence that thedesign outputs of a particular phaseof the software development life cyclemeet all of the specified requirementsfor that phase.ValidationConfirmation …… that softwarespecifications conform to user needs<strong>and</strong> intended uses, <strong>and</strong> that theparticular requirements implementedthrough software can be consistentlyfulfilled….Since software is usuallypart of a larger hardware system, thevalidation … includes evidence thatall software requirements have beenimplemented correctly <strong>and</strong> completely<strong>and</strong> are traceable to systemrequirements.Sources:Regulation: Code of Federal Regulations 21 Part 830, Subpart CDesign ControlsGuidance: General Principles of <strong>Software</strong> Validation© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.15
AndCode of Federal Regulations CFR 21 Part 830, Subpart C Design Controlsm<strong>and</strong>ates a requirements specification.Requirements SpecificationA documented software requirementsspecification (SRS) provides a baseline forboth validation <strong>and</strong> verification. Thesoftware validation process cannot becompleted <strong>with</strong>out an established softwarerequirements specification(Ref: 21 CFR 820.3(z) <strong>and</strong> (aa) <strong>and</strong> 820.30(f)<strong>and</strong> (gTraceabilityFDA guidelines describe traceability <strong>and</strong> aprimary mechanism to assure thatverification <strong>and</strong> validation are complete<strong>and</strong> consistent.Traceability. The degree to which arelationship can be established betweentwo or more products of the developmentprocess, especially products having apredecessor-successor or mastersubordinaterelationship to one another;e.g., the degree to which the requirements<strong>and</strong> design of a given software componentmatch [IEEE]Sources:Regulation: Code of Federal Regulations 21 Part 830, Subpart CDesign ControlsGuidance: General Principles of <strong>Software</strong> Validation© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.16
A High Assurance, <strong>Agile</strong> <strong>Software</strong><strong>Development</strong> Lifecycle<strong>Development</strong> SprintsValidation Sprints© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.17
High Assurance Scaled <strong>Agile</strong> Framework ©2011 Leffingwell LLC.System increment System increment Valida6on Valida6on Valida6on H H Valida6on Valida6on Valida6on H H See <strong>Agile</strong> So)ware Requirements: Lean Requirements Prac8ces for Teams, Programs, <strong>and</strong> the Enterprise <strong>and</strong> www.scalingso/wareagility.wordpress.com
ITERATING AND CONTINUOUSVERIFICATION19
<strong>Agile</strong> Itera6on Pa:ern
The User StoryUserStoryAs a I can So that As an EPAT (Extracorporeal Pulse Activation Technology) technician,() I can adjust the energy delivered ()in increments so as to deliver higher or lower energy pulses to thepatient’s treatment area ().© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.
View of an Iteration© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.22
High assurance agile requirementsmodelSource: Leffingwell. <strong>Agile</strong> <strong>Software</strong>Requirements: Lean Requirements Practicesfor Teams, Programs, <strong>and</strong> the Enterprise.Addison-Wesley 2011.© 2011 <strong>Rally</strong> <strong>Software</strong> © 2011 <strong>Development</strong> Leffingwell, <strong>and</strong> LLC. Leffingwell, LLC.
Traceability from User Story to Code <strong>and</strong>Story Acceptance Test<strong>Software</strong>RequirementsSpecificationUser StoryImplemented by11..*1 1CodeVerified byVerified by1..*1..*Unit TestStoryAcceptance Test© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.
Traceability from User Story to Code© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.25
Traceability from User Story to Acceptance Test© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC. 26
VALIDATING SYSTEMINCREMENTS© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.27
Validating product claimsProductIncrementsFeature1Product RequirementsDocumentPulse amplitude is adjustablefrom 1-5 barTraced to1..*Story<strong>Software</strong>RequirementsSpecificationAs an operator, I can adjust the pulseamplitude in .1 bar increments so as to beable to make small changes to changeenergy delivered to patient areaAs an operator, I alwayssee the current settingon the display in .1 barincrements, so I can beconfident I’m deliveringthe right energyAs an operator, rotating the energy knob pastthe point where the system is delivering 5 barwill have no further effect© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.
Supporting product claims to implementation© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC. 29
Validating Features <strong>and</strong> System Qualities© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.30
Validation Sprint activities© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.31
© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC. 32
© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC. 33
(Michael’s slides)IMPACT ON QUALITYMANAGEMENT SYSTEMS© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.34
Important points to reiterate <strong>Agile</strong> <strong>and</strong> FDA are not at odds (waterfall is not m<strong>and</strong>ated by regulations) Internal interpretation of regulations often more constraining <strong>Agile</strong> extremism does not help (working software over documentation) Satisfy Compliance needs while preserving agility <strong>and</strong> productivity. <strong>Agile</strong> <strong>and</strong> Regulations both strive to ensure high product quality© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.35
Quality System (QS) Continuous improvement or (re-)write from scratch Establish cross-functional QS scrum team Run releases <strong>and</strong> sprints to refine / establish QS Design Controls needs to provide flexibility <strong>Software</strong> <strong>Development</strong> Life Cycle (SDLC), Tools, etc. should bespecified in the Design <strong>and</strong> <strong>Development</strong> Plan (DDP), not in the QS© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.36
Change ControlsNew Product <strong>Development</strong>: Establish initial product backlog (PRD, SRS, DDP, etc.) H<strong>and</strong>le Risk Assessment, FMEAs, etc. outside scrum teams Develop features through sprints <strong>and</strong> “releases” File deviations per “release” using Engineering Change Order (ECO)Update release or patch: File intended changes using Engineering Change Order (ECO) Develop the features / fix relevant defects Revise the ECO along the way, if changes to plan occurred© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.37
<strong>Software</strong> <strong>Development</strong> Life CycleDownloadpresentation slides© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.38
CFR 820.11 Compliance <strong>and</strong> Tool Validation Tools used to manage Design History File (DHF) artifacts need tobe CFR 820.11 compliant Tool Validation: Establish objective evidence that a tool consistentlymeets the specifications it is being used for If you use RALLY or any other agile tool for such purpose then youneed to validate it For any SAAS vendor, negotiate controlled software releases toyour account or host your solution in-house© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.39
Revision Control across releases If you have all requirements, specifications, defects, test cases, etc. inan agile tool, how do you manage changes to past releases so thatyou can re-run the right design verification tests from back then? You are better off to manage past releases outside the agile tool <strong>and</strong>only keep latest greatest revision in agile tool Keeping artifacts of past release externally not a major issue© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.40
Q&A© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.41
More Books– <strong>Agile</strong> <strong>Software</strong> Requirements: Lean Requirements Practicesfor Teams, Programs, <strong>and</strong> the EnterpriseAddison-Wesley, Leffingwell. 2011.– Scaling <strong>Software</strong> Agility: Best Practices for Large Enterprises.Addison-Wesley, Leffingwell. 2007. Leffingwell Blog– www.scalingsoftwareagility.wordpress.com (high assurancecategory) <strong>Rally</strong> High Assurance Blog– http://www.rallydev.com/agileblog© 2011 <strong>Rally</strong> <strong>Software</strong> <strong>Development</strong> <strong>and</strong> Leffingwell, LLC.42
Change language