Developing medical devices that are subject to FDA approval has traditionally followed the waterfall methodology, largely due to the structure of the regulations that govern development practices. But we know from myriad case studies in different industries that agile methodologies are far superior in providing the highest value to customers in the shortest time to market. Neal Herman shares how one developer of complex medical devices embraced agile software development practices and proved that it could not only develop software faster with higher quality but also meet all regulatory requirements. Convincing the internal quality management, systems engineering, and regulatory departments was difficult, but the software department was able to overcome these obstacles and fundamentally change the company’s philosophy on product development. Since 2012, software development productivity is up 100 percent, and quality is up 200 percent. Now, after seeing these gains from the software department, agile is being rolled out to all areas of R&D including hardware.

1. AT2
Session
6/9/16 10:00 AM
Implementing Agile in an FDA
Regulated Environment
Presented by:
Neal Herman
BD Biosciences
Brought to you by:
350 Corporate Way, Suite 400, Orange Park, FL 32073
888---268---8770 ·· 904---278---0524 - info@techwell.com - http://www.techwell.com/

2. Neal Herman
BD Biosciences
Neal Herman is the senior manager of software and firmware quality for BD
Biosciences, and the Chief ScrumMaster and Agile Coach for the R&D
department. Neal regularly consults with other R&D organizations within the
parent company and recently organized a company-wide agile conference and
workshop. In addition to Scrum, his teams employ other software development
best practices including CI, TDD, and pair programming. BD Biosciences uses
kanban as a scaling mechanism to track their product backlogs at the portfolio
level. Neal has more than thirty years' experience as a software development
manager in a variety of domains—logistics, RFID, motion control, factory
automation and robotics, artificial intelligence, and commercial banking.

3. 5/17/16
1
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
Implementing Agile in an
FDA Regulated
Environment
Neal Herman
Sr. Manager of Firmware and
Software Quality
BD Biosciences
June 9, 2016
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
Neal Herman’s Bio
• Sr. Manager of Firmware and Software Quality
• Chief Scrum Master and internal Agile coach
• Involved in software development for 30+ years as a
developer, project lead, and manager
• Domain expertise in financial systems, AI (expert systems),
factory automation and robotics (motion control), RFID,
defense logistics, and medical devices
• Started at BD in January 2011
• Past companies include Xerox, Schering-Plough, IBM,
Syntelligence, Galil Motion Control, and Savi Technology
(Lockheed Martin)
• Certified Scrum Master and Certified Scrum Professional
(Scrum Alliance), Professional Scrum Master I (Scrum.org)
• Practicing Scrum since 2009
2

4. 5/17/16
2
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
• Introduction to BD Biosciences
• Why Agile?
• What obstacles prevented us from implementing Agile and
how did we overcome them?
• What does the FDA say about Agile?
• What obstacles prevented us from implementing Agile
effectively and how did we overcome them?
• What factors helped us with a successful transition to Agile?
• Roll-out of Agile in the R&D organization beyond software
• Q&A
Agenda
3
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
Part of BD Life Sciences business segment
More than $1 billion in revenue
Producer of cell analyzers, cell sorters, and reagents for the
research and clinical markets
General applications:
• Immunology
• Drug discovery
• Cell therapy enablement
• General research for biological cells (intracellular and extracellular)
• Genomics sample enrichment and library prep
BD Biosciences – who we are
4

5. 5/17/16
3
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.5
Instruments
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
Reagents
6

6. 5/17/16
4
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
Software
7
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.8
Four-way
sort at
25,000 events/
second
Example Application: Four-Way Sort Mouse
Spleen Cells

7. 5/17/16
5
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
Waterfall approach with defined phase-gates not meeting our
needs
• Time to market too long
• Products not meeting customer expectations
• Poor quality
• At least for version 1.0 of instruments and software
• Difficult to track progress and estimate program completion
Why Agile?
9
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
• Waterfall, i.e., sequential development
• All requirements, including software under design control
before start of development
• Phase Gates with formal entry/exit criteria
• Long test cycles
• Sub-system verification
• System verification
• System validation
• Very long elapsed time between requirements definition
phase and validation phase
• At the end of all this, did we produce the right product with
the right feature set?
Traditional Product Development
at BD
10

8. 5/17/16
6
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
The V-Model
11
• Traditional Development for Medical Devices
• Sequential development, defined entry/exit
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
• Resistance from Quality and Regulatory departments
• “Our SOPs mandate a Waterfall approach”
• “The FDA doesn’t allow it”
• Resistance from R&D leadership who were comfortable with
the old approach
• Voluminous documentation from FDA and other governing
organizations creating confusion about what is actually
required or allowed
What was Preventing us from
using Agile?
12

9. 5/17/16
7
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
References from General Principles of Software Validation; Final
Guidance for Industry and FDA Staff
• Design Control Guidance for Medical Device Manufacturers, Center for Devices and Radiological Health, Food and Drug Administration, March 1997.
• Do It by Design, An Introduction to Human Factors in Medical Devices, Center for Devices and Radiological Health, Food and Drug Administration, March 1997.
• Electronic Records; Electronic Signatures Final Rule, 62 Federal Register 13430 (March 20, 1997).
• Glossary of Computerized System and Software Development Terminology, Division of Field Investigations, Office of Regional Operations, Office of Regulatory Affairs, Food and Drug Administration,
August 1995.
• Guidance for the Content of Pre-market Submissions for Software Contained in Medical Devices, Office of Device Evaluation, Center for Devices and Radiological Health, Food and Drug Administration,
May 1998.
• Guidance for Industry, FDA Reviewers and Compliance on Off-the-Shelf Software Use in Medical Devices, Office of Device Evaluation, Center for Devices and Radiological Health, Food and Drug
Administration, September 1999.
• Guideline on General Principles of Process Validation, Center for Drugs and Biologics, & Center For Devices and Radiological Health, Food and Drug Administration, May 1987.
• Medical Devices; Current Good Manufacturing Practice (CGMP) Final Rule; Quality System Regulation, 61 Federal Register 52602 (October 7, 1996).
• Reviewer Guidance for a Pre-Market Notification Submission for Blood Establishment Computer Software, Center for Biologics Evaluation and Research, Food and Drug Administration, January 1997
• Student Manual 1, Course INV545, Computer System Validation, Division of Human Resource Development, Office of Regulatory Affairs, Food and Drug Administration, 1997.
• Technical Report, Software Development Activities, Division of Field Investigations, Office of Regional Operations, Office of Regulatory Affairs, Food and Drug Administration, July 1987.
• ANSI / ANS-10.4-1987, Guidelines for the Verification and Validation of Scientific and Engineering Computer Programs for the Nuclear Industry, American National Standards Institute, 1987.
• ANSI / ASQC Standard D1160-1995, Formal Design Reviews, American Society for Quality Control, 1995.
• ANSI / UL 1998:1998, Standard for Safety for Software in Programmable Components, Underwriters Laboratories, Inc., 1998.
• IEC 60601-1-4:1996, Medical electrical equipment, Part 1: General requirements for safety, 4. Collateral Standard: Programmable electrical medical systems. International Electrotechnical
Commission, 1996.
• IEC 61506:1997, Industrial process measurement and control - Documentation of application software. International Electrotechnical Commission, 1997.
• IEC 61508:1998, Functional safety of electrical/electronic/programmable electronic safety-related systems. International Electrotechnical Commission, 1998.
• IEEE Std 1012-1986, Software Verification and Validation Plans, Institute for Electrical and Electronics Engineers, 1986.
• IEEE Standards Collection, Software Engineering, Institute of Electrical and Electronics Engineers, Inc., 1994. ISBN 1-55937-442-X.
• ISO 8402:1994, Quality management and quality assurance - Vocabulary. International Organization for Standardization, 1994.
• ISO 9000-3:1997, Quality management and quality assurance standards - Part 3: Guidelines for the application of ISO 9001:1994 to the development, supply, installation and maintenance of
computer software. International Organization for Standardization, 1997.
• ISO 9001:1994, Quality systems - Model for quality assurance in design, development, production, installation, and servicing. International Organization for Standardization, 1994.
• ISO 13485:1996, Quality systems - Medical devices - Particular requirements for the application of ISO 9001. International Organization for Standardization, 1996.
• ISO/IEC 12119:1994, Information technology - Software packages - Quality requirements and testing, Joint Technical Committee ISO/IEC JTC 1, International Organization for Standardization and
International Electrotechnical Commission, 1994.
• ISO/IEC 12207:1995, Information technology - Software life cycle processes, Joint Technical Committee ISO/IEC JTC 1, Subcommittee SC 7, International Organization for Standardization and
International Electrotechnical Commission, 1995.
• ISO/IEC 14598:1999, Information technology - Software product evaluation, Joint Technical Committee ISO/IEC JTC 1, Subcommittee SC 7, International Organization for Standardization and
International Electrotechnical Commission, 1999.
• ISO 14971-1:1998, Medical Devices - Risk Management - Part 1: Application of Risk Analysis. International Organization for Standardization, 1998.
• Software Considerations in Airborne Systems and Equipment Certification. Special Committee 167 of RTCA. RTCA Inc., Washington, D.C. Tel: 202-833-9339. Document No. RTCA/DO-178B, December
1992.
Small sample of documents
referenced in FDA guidance
13
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
How do I digest all this
information?
14

10. 5/17/16
8
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
Just a small fraction of these documents provide all the
relevant guidance for developing software for medical devices
Making sense of all the available
documents/information
15
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
Primary guidance documents
16
Document Descrip.on
ANSI/UL 1998
Software in Programmable Components
62304 First edition 2006-05
Medical device software - Software life cycle processes
AUTO13-A2 (Formerly GP19-A2)
Laboratory Instruments and Data Management Systems: Design of
Software User Interfaces and End-User Software Systems Validation
Operation and Monitoring; Approved Guideline - Second Edition
AUTO11-A
IT Security of In Vitro Diagnostic Instruments and Software Systems;
Approved Standard
62304:2006
Medical device software - Software life cycle processes
TIR 36:2007
Validation of software for regulated processes
TR80002-1 Edition 1.0 2009-09
Medical device software - Part 1: Guidance on the application of ISO
14971 to medical device software
TIR45:2012
Guidance on the use of AGILE practices in the development of medical
device software
15026-4 First Edition 2012-10-01
Systems and software engineering - Systems and software assurance -
Part 4: Assurance in the life cycle
1998 Third Edition 2013
Standards for Safety Software in Programmable Components Second
Edition. [This Standard contains revisions through and including
October 28 2008]

11. 5/17/16
9
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
Primary guidance documents
17
Document Descrip.on
62304 First edition 2006-05
Medical device software - Software life cycle
processes
TIR45:2012
Guidance on the use of AGILE practices in the
development of medical device software
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
Primary guidance documents
18
Document Descrip.on
62304 First edition 2006-05
Medical device software - Software life cycle
processes
TIR45:2012
Guidance on the use of AGILE practices in the
development of medical device software

12. 5/17/16
10
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
Agile and the FDA
• FDA officially recognizes Agile as a valid software
development methodology for medical devices
• Guidance document AAMI TIR 45:2012
– Recognized Consensus Standard
• Key takeaway: design inputs must be synchronized with
design outputs at the end of each sprint or iteration
– Requirements, design specifications, test procedures, code base, etc.
must be in sync
– No prescribed order meaning requirements can be generated at the end
of the sprint if that makes more sense
19
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
Agile and the FDA
Synchronizing DESIGN INPUT/OUTPUT at SPRINT and RELEASE boundaries
20 Source: AAMI TIR45:2012

13. 5/17/16
11
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
Agile and the FDA
Mapping IEC
62304’s activities
into AGILE’s
INCREMENTAL /
EVOLUTIONARY
lifecycle
Source: AAMI TIR45:2012
21
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
The V Model revisited
22
Every sprint we verify and validate
• Did we do the thing right?
• Did we do the right thing?

14. 5/17/16
12
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
Its all about the requirements
23
• Managing requirements using an Agile approach is
conducive to creating a better product
• System requirements are not transformed into working
product until the last responsible moment
• Collaboration between stakeholders and development
teams throughout the project, not just at the Definition
phase
• Involve all SMEs including system engineers, verification
and validation specialists, field service, and other
downstream stakeholders
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
Its all about the requirements
24
System
Requirement
Feature
Epic
Story
Story
Epic
Story
Story
Feature
Epic
Story
Story
Epic
Story
Story

15. 5/17/16
13
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
User stories connect all the dots
25
Test Cases
• Acceptance
Criteria
• Requirement
VerificaJon
Development
• Unit Tests
• Code Reviews
• Code Check-ins
• Design
Documents
• User
DocumentaJon
User
Story
System
Requirement
DocumentaJon
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
• Too many SOPs
• Many aligned with Waterfall
• Most were overly prescriptive and did not provide a lot of flexibility
What was preventing us from
using Agile effectively?
26

16. 5/17/16
14
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
• Reduced the number of SOPs and related documents
• Demoted many documents from “must do” to “guidance”
• Made the remaining SOPs more flexible and harmonious
with Agile development practices
How did we address SOP issue?
27
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
Standard Operating Procedure (SOP):
• A small, meaningful set of documents
• Only what’s minimally required for compliance
Desk Procedure (DP):
• Describe “how-to” processes minimally required for compliance
Guidance Master (GM)*:
• Procedures, templates, checklists, and other tools to provide guidance
and help achieve internal business objectives
• Flexible from an auditor’s perspective, but can be enforced by
functional managers or core teams
• Reports to a parent SOP or DP that defines the truly mandatory aspects
of the process (same base number as parent)
*New document type
Leaning R&D SOPs
28

17. 5/17/16
15
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
• Some SOPs demoted to DPs or GMs
• Some DPs demoted to GMs
• Eliminated 87 documents
Leaning R&D SOPs
29
Before Streamlining A8er Streamlining
SOPs 173 17
DPs 76 46
GMs 0 99
Total 249 162
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
• Stick closely to the rules of Scrum
• Integrate Development and Test
• Create a “Definition of Done” and adhere to it
• Deal with problems as they materialize
• Make all work visible
• Work on one task at a time
• Spend as much time grooming the Product Backlog as is
necessary
• Publish a Dashboard and make sure it reflects the truth
• Create a consistent process across all Scrum teams
• Provide proper training and coaching
• Secure management support
What factors helped us with a
successful transition to Agile

18. 5/17/16
16
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
• Rules of team membership, Scrum roles, Scrum
ceremonies, Scrum artifacts
• You can add other Agile practices on top of Scrum such as
Lean and Kanban but never subtract
• Especially true for an organization new to Scrum
Stick Closely to the Rules of
Scrum
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
• Developers and testers must collaborate on story grooming
and generating acceptance criteria
• Testing tasks are not just for testers
– Developers must be willing to accept testing tasks if that’s what's
necessary to close a story
– Optimize globally, not locally – its not about keeping developers 100%
utilized on coding, its about completing stories
• Consider BDD testing techniques such as Cucumber
Integrate development and test

19. 5/17/16
17
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
• This is your key quality gate
• User stories must satisfy both user acceptance criteria and
technical checklist
• Stories are not “done” if technical debt is being
accumulated
– Will only make your Release Burndown untrustworthy
• Create a checklist that closely conforms to IEC 62304
development activities
– Design inputs synchronized with design outputs
Create a “Definition of Done” and
adhere to it
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
Checklist for Technical Completeness for User Stories
Design:
• Design covers everything in the user story and acceptance criteria.
• Design reviewed by area experts and feedback is incorporated.
• User story has a link to the design.
Code:
• Code implements the design.
• Unit tests cover the design (includes use cases, API contracts).
• Code compiles and runs, on the build server, without errors, warnings, or unit test failures.
• Code and unit tests have been peer reviewed and adjustments made per comments.
• No new defects.
Acceptance Testing:
• Acceptance tests in a form listed below have been written and entered into project management system.
– May include manual, automated, and unit or integration tests
– Verification Procedure and/or SMART for high risk stories
– SMART for stories with medium risk
• Acceptance tests have been reviewed by a developer and feedback has been incorporated.
• Acceptance tests pass on a branch or main build; unresolved issues found on main build have been logged into defect
tracking system.
• Executed results have been attached to project management system.
Defect Fixing:
• Minimal steps to reproduce are documented in the defect description.
• Root cause analysis is documented in the defect description.
• Fix approach is documented in the defect description.
Definition of Done
34

20. 5/17/16
18
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
• Scrum is very good at exposing problems
• Make all problems visible and track them
• Each problem being tracked should have a responsible
person and a due date
• Discuss at team and/or project stand up
Deal with problems as they
materialize
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
• Use a white board, Post-it® notes, electronic board, etc.
– Team members must update time remaining daily
• Track both Scrum and non-Scrum tasks
• Use more advanced Kanban boards to expose workflow
bottlenecks
Make all work visible

21. 5/17/16
19
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
• Complete tasks (or stories) before starting new ones
• Excessive multi-tasking and WIP inhibits throughput
Work on one task at a time
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
• Complex projects may need many hours to properly groom
the Product Backlog and individual user stories
• Make sure that user stories are ready for teams to work on
before they are accepted into a sprint
– Avoids costly re-work due to misunderstanding of requirements
Spend as much time grooming the
product backlog as is necessary

22. 5/17/16
20
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
• Make sure your product backlog is estimated using effort
not time
– Estimates should be made in a consistent way
• Measure velocity by only counting stories that meet the
“Definition of Done”
• The resulting Release Burndown will prove to be an
accurate (but not precise) indicator of when the software
release will be complete
Publish a dashboard and make
sure it reflects the truth
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
FACSuite IVD – Release Dashboard
40
Note: the original
project scope was
11000 story points
– this chart only
shows the tail of
the project

23. 5/17/16
21
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
FACSuite IVD – FCAT Dashboard
41
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
• Without a consistent process, scaling becomes more
difficult
• Allows team members to more easily move from team to
team
• Eases the burden for training and coaching
Create a consistent process across
all Scrum teams

24. 5/17/16
22
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
• Scrum isn’t hard to understand but it’s difficult to
implement properly
• Everyone in the organization needs to be on the same page
including managers
– But don’t forget Quality, Systems, Regulatory, Marketing, Program
Management, etc.
Provide proper training and
coaching
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
• Line/functional managers must not micro manage Scrum
teams
– Do not disturb teams during a sprint
• Leadership must not use estimates and metrics (story
points, velocity) to reward or punish individuals or teams
• Management must allow some time for Scrum teams to
develop and mature
– Results take time but they will come
Secure management support

25. 5/17/16
23
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
After the successes with the Software Department, BD
Biosciences is implementing Scrum across the organization
• Hardware
• Systems Engineering
• Systems Verification & Validation
• Reagents
• Program level backlog that decomposes into sub-system
backlogs
• Program-level Product Owner
– Coordinates activities of sub-system POs
• Daily stand-up for at the program level that includes all POs
and SMs
Agile: not just for software
45
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
Questions?
46

26. 5/17/16
24
© 2016 BD. BD, the BD Logo and all other trademarks are property of Becton, Dickinson and Company.
Thank you!
47