Elements of an effective analytics strategy and tips for working with a clinical research organization to develop a metric system.

1. 1| Confidential and Proprietary
Implementing a Metrics System
with a CRO Partner
06 Dec 2016
Mike Fitzpatrick
mfitz0610@gmail.com
@fitzviz
These slides and the content of this presentation represent the work and opinions of the author and do not
constitute official positions of Biogen or any other organization. This document provides an outline of a
presentation and is incomplete without the accompanying oral commentary and discussion. Graphics,
figures and data contained herein are for illustrative purposes and are not representative of actual data.

2. 2| Confidential and Proprietary
Today’s Plan
• Intro
• A Case for Analytics
• Analytics Journey
• Elements of an Effective Analytics Strategy
• Design and Story Telling
• Metric Development Framework
• Wire Framing
• Rapid Prototyping
• Chart Selection
• Metrics for your metrics
• Tips for Working with A CRO

3. 3| Confidential and Proprietary
Qualifications:
• BS Industrial Engineering,
Management Minor from WPI
• Lean Six Sigma Green Belt
• Tableau Qualified Associate
About Me
Interests:
• Data visualization and analytics
Blog: www.fitzanalytics.com
Twitter: @fitzviz
• Real Estate
Company: www.fitzproperty.com
Current Job
Biogen Clinical Operations
Performance Capabilities:
• Process Improvement
• Metrics and Analytics
• Clinical Systems
Prior Experiences
• Biogen Clinical Supply Chain
• Biogen Co-Monitor (CRA)
• Boston Scientific
• ATC Consulting Group
• Nypro (a Jabil Company)
Resume Viz

4. 4| Confidential and Proprietary
Why not for clinical trials?
https://www.youtube.com/watch?v=V1eYniJ0Rnk

5. 5| Confidential and Proprietary

6. 6| Confidential and Proprietary
“The ability to take data – to be able to understand
it, to process it, to extract value from it, to visualize
it, to communicate it’s going to be a hugely
important skill in the next decades”
-Google’s Chief Economist Hal Varian – 2009 Interview
Our Job

7. 7| Confidential and Proprietary
DATA DATA DATA
DATA
DATA
INTEGRATED DATA SOURCE = “single source of the truth”
VISION
LEADERSHIP
SUPPORT
PAST (2013-2015):
• Many tools, lots of ad-hoc reporting
• Better understand needs and gaps
• Develop Key Performance Indicators
PRESENT:
• Refining Key Performance Indicators
• Develop metrics based on needs
• Minimize number of analytics tools
• Improve data quality and sources
FUTURE:
• Well known KPIs, Deep Learning
• Integrated data and data governance
• Shared best practices
• Single location for analytics
Goal ManagementCycle Times
Risk Assessment
Lots of Development

8. 8| Confidential and Proprietary
Design Philosophy
Source: http://cargocollective.com/central/The-Design-Squiggle
Research Concept Design Ideas
“Graphical excellence is that which gives to
the viewer the greatest number of ideas in
the shortest time with the least ink in the
smallest space.” – Edward Tufte
“The purpose of
visualization is
insight, not pictures.”
– Edward Tufte.

9. 9| Confidential and Proprietary
Importance of Data Storytelling
Source: Brent Dykes - Forbes

10. 10| Confidential and Proprietary
Methodology to Insightful Metrics
Focus on
important
areas (CSF)
Implement
“closed
loop”
process for
action on
insights
Identify Key
Questions to
be answered
Recognize
related
actions
Consider
driving
behaviors for
a metric
Define the
metric:
• Available
data
• Trustworthy
data
• Targets
• Frequency
• Owners
Before jumping to metrics definition, we
need to look at these first !!

11. 11| Confidential and Proprietary
Steering Committee
Critical Success FactorCritical Success Factor Critical Success FactorCritical Success Factor Critical Success Factor
Metric
Metric
Metric
Metric
Metric
Metric
Metric Metric
Metric
Metric
Metric
Metric
Strategy Mapping What does your
governance
framework look like?

12. 12| Confidential and Proprietary
Create Wireframes to Inspire
Feedback and Ideas

13. 13| Confidential and Proprietary
Just do it  Prototype Rapidly
Source: Andy Kriebel #MakeoverMonday

14. 14| Confidential and Proprietary

15. 15| Confidential and Proprietary
Chart Selection Example
Sources:
Save the Pies for Dessert Stephen Few, Perceptual Edge Visual Business Intelligence Newsletter August 2007
http://public.tableau.com/profile/brianhalloran#!/vizhome/FunnyChartsTableau-ed/PieCharts

16. 16| Confidential and Proprietary
Metrics for Your Metrics

17. 17| Confidential and Proprietary
Strategy/Culture
• Understand governance forums
where metrics will be reviewed
• Create a strategy map to align both
companies with leadership support
Logistics
• Create an analytics workspace
(sharepoint, wiki etc)
• Develop simple templates to report
metric status and efforts to improve
• Track and trend action items
Training/Coaching
• Identify metric owners and sponsors
• Teach people best practices for data
story telling and metrics development
Tips for Working with a Partner/CRO
Brainstorming and Prioritizing Metrics
• Listen and take notes, don’t curb
creativity, get it all out there
• Expect a large set of “wish-list”
metrics and keep it handy
• Use 3rd parties to help gain influence
and prioritize the “critical few”
Data Requests and Development
• Be clear with data ask (wire frame)
• Expect data quality issues and plan
to allocate time for troubleshooting
and data clean up
• Track your deliverables and have a
sustainment plan for on-going
management of dashboards/metrics

18. 18| Confidential and Proprietary
10 mins – Executive Summary
10 mins – Review Strategy Map
10 mins – Acknowledge Accomplishments
30 mins – Review At-Risk and Off-Track
Note: Details of each metric status and next actions are
provided as a pre-read document and action item tracker
Example Agenda Governance
Review Meeting (Quarterly)

19. 19| Confidential and Proprietary
Example of CRO Strategy Map
Strategic Oversight Areas
Time Quality Compliance Financial
Customer
Service
Partnership
Health
ESC
JOC
TIME METRIC
QUALITY
METRIC
COMPLIANCE
METRIC
CUST SERVICE
METRIC
PSHIP HEALTH
METRIC
TIME METRIC
QUALITY
METRIC
COMPLIANCE
METRIC
FINANCE
METRIC
CUST SERVICE
METRIC
PSHIP HEALTH
METRIC
TIME METRIC
QUALITY
METRIC
FINANCE
METRIC
PSHIP HEALTH
METRIC
COMPLIANCE
METRIC
FINANCE
METRIC
TIME METRIC

20. 20| Confidential and Proprietary
Example of Governance Reporting
Template for At-Risk and Off-Track
Metric Name Metric Status Efforts to Improve
OFF
TRACK
Metric Name Metric Status Efforts to Improve
AT RISK
Separate table
for each metric

21. 21| Confidential and Proprietary
Standardize the tracking of:
• Date (Assigned , Due and Completed)
• Metric Name
• Type of Issue (data quality, study
performance, definition, target, etc)
• Effort (H/M/L)
• Impact (H/M/L)
• Owner
• Issue Description
• Status/Resolution
Example of Action Item Tracking
Use tracker to produce oversight metrics:
• Overall status of actions
• # actions by type of issue
• # actions by metric
• # action by owners
• # action overtime
• # action by due date
Metrics have a shelf-life use this data to
justify changes to metrics and continuous
improvement

22. 22| Confidential and Proprietary
Industry:
• Metrics Champion Consortium
• Pharmaceutical Metrics: Measuring and Improving R & D Performance
• Tufts Center for the Study of Drug Development
Data Visualization:
• Tufte Presenting Data and Information Course
• http://flowingdata.com/
• http://www.thefunctionalart.com/
• http://polygraph.cool/
Resources

23. 23| Confidential and Proprietary
Contact Info
Mike Fitzpatrick
@fitzviz
mfitz0610@gmail.com