When Perfect Algorithms Meet Imperfect Healthcare Systems: My talk at the Machine Learning for Healthcare Meeting 2018

1. When
Perfect Algorithms
Meet
Imperfect Healthcare Systems
Joyce Lee, MD, MPH
Robert Kelch Professor of Pediatrics
doctorasdesigner.com
@joyclee on Twitter

2. Lenovo (Grant Funding)
Disclosures
@joyclee

3. www.doctorasdesigner.com
medium.com/@joyclee
Professor, Robert Kelch Professor of Pediatrics
Clinical/Health Outcomes Research
(Obesity/Diabetes)
Learning Health Systems
(Clinical Informatics, Quality Improvement, Participatory Design)
Applications of Emerging Technologies in Healthcare
(Ethnography, Analysis of Diabetes Online Maker Communities)
@joyclee

4. @joyclee
Agenda
Why Diabetes Data Matters
Diabetes Data Barriers
Patient Innovation with Data

5. Why
Diabetes Data
Matters
@joyclee

6. @joyclee
It’s an infinite amount of math
over a lifetime

7. It’s an infinite amount of math
over a lifetime
@joyclee

9. Blood Glucose
Meters
Continuous Glucose
Monitoring (CGM)
Systems
Insulin Pumps

10. @joyclee
Glucose levels change all the time
Puberty Growth Activity

11. Glucose levels impact health outcomes

12. Management is truly patient-driven
Provider Time
managing diabetes
(1 hour in a year?)Patient Time Spent
Managing Diabetes

13. Data Nirvana for Cars
@joyclee
“The Autonomous Car”

14. The Singularity!
@joyclee

15. @joyclee
“The Artificial Pancreas”
Continuous
Glucose Monitor
Insulin
Pump
Mobile Phone
with Algorithm
Data Nirvana for Diabetes

16. The Singularity!
Goodbye Pediatric Endocrinologist!
You have been eliminated!
@joyclee

17. The Singularity!
…not
really
@joyclee

18. @joyclee
“The Artificial Pancreas”
“Closing the Loop"
Medtronic 670G DIY In Development

19. What are the
Diabetes Data
Barriers?
@joyclee

20. No data
Trapped data
Infrequent data
Isolated data
Confusing data
Biased data
Judgmental data
Suspicious data @joyclee

21. No data
@joyclee

22. We still give out mostly analog tools
”It was unacceptable to me in 2002, when my son was
diagnosed, to be given needles, an insulin vial, and a piece
of paper.”
- Jeff Brewer, Bigfoot CEO @joyclee

23. Blood Glucose Meters (100%)
Insulin Pumps (50%)
Continuous Glucose Monitoring (CGM) Systems (15%)
Connected Pens (<1%)
AP Systems (<1%)
Device Adoption Rates (Type 1 Diabetes)
@joyclee

24. Diabetes tools are very expensive
$1411 Starter Kit
$596 Transmitter (every 3 months)
$349/month for Sensors
$70-80 Starter Kit
$120-150/month for Sensors
@joyclee

25. Insurance companies don’t like to pay
for things like blood glucose test
strips, connected pens, and CGMs

26. Burnout +
Stigma +
Burden +
Frustration
= No BG data
@joyclee

27. Trapped data
@joyclee

28. Email to fax:
xxxxxxx-diabetes-fax@med.umich.edu
@joyclee

29. @joyclee

30. @joyclee

31. People Prefer Mobile to Desktop
@joyclee

32. Infrequent data
@joyclee

33. Downloads and diabetes decision-
making only happens at clinic
3-4 visits per year with 15
minutes per visit spent on
data
Meter/pump/CGM is
collected at the clinic visit
and a data download is
exported as a PDF and
scanned into the media tab
of the chart
@joyclee

34. Mental Health
Issues
20,028 calls in 2017
@joyclee
School Forms
Supplies
Prior Authorization
Insurance
Child Protection
Services
DME
Pharmacy

35. Patient data review outside of clinic
is reactive, not proactive
Reporting blood
glucose numbers
over the phone
PDF attached
to a Portal Message
(page limits)
Fax (email-fax)
48 hour turnaround time

36. Isolated data
@joyclee

37. @joyclee

38. We get upset when the Color Printer
is not working in clinic
@joyclee

39. Confusing data
@joyclee

40. Technology is about culture
change
”We’re living through this time right now where
technology is a Trojan Horse for change. We say
technology, but we mean innovation. We say
interoperability and open data, but we mean
culture change.”
-Susannah Fox
@joyclee

41. Onboarding Patients to the
Patient Portal and Diabetes Data
Platforms is no one’s job
Not the Health IT Specialists
Not the Medical Assistants
Not the Educators
Not the Doctors
@joyclee

42. There is no formal patient education
focused on how to download data,
how to interpret it, and how to use it
to adjust insulin doses
“Am I supposed to still keep a logbook?”
“I don’t feel comfortable making dose adjustments
without first consulting with the CDE or
endocrinologist. I mean, it’s his life, you know?”
@joyclee

43. “Routine Downloaders” = Downloaded data at least once between
routine clinic visits every 3 months which was four or more times
in the past year (40%)
“Routine Reviewers’’ = Reviewed the data at least ‘‘most of the
time’’ he/she downloaded (27%)
Lower A1c for Routine Reviewers: 7.8% vs. 8.6% (p=0.001)
Diabetes Data Platform Use
Wong et al, 2015
@joyclee

44. There is a lack of standardization for
data visualizations
@joyclee

45. @joyclee
There is a lack of standardization for
data visualizations

46. Biased data
@joyclee

47. @joyclee
Are they covering all carbs?
Are they carb counting carefully?
Are they inputting all the carbs / BG values into the pump?
Are they bolusing before or after meals?
Can I trust the data?
Insulin lasts 2-3 hours
There are A Lot of Unmeasured Variables

48. Judgmental data
@joyclee

49. Guilt
Loss of Control
Alienation
Obsession
Data is not always empowering
Isolation
@joyclee
CSCW 2017, Kaziunas, Ackerman, Lindtner, Lee

50. Suspicious data @joyclee

51. Patients and clinicians don’t trust a
black box
“I don’t know what
it’s doing so how
can I trust it?”
“I wouldn’t give up
my DIY AP”
@joyclee

52. And that doesn’t
even include linking this vital
diabetes data to the
clinical EHR data!
@joyclee
“Garbage data”

53. Lack of User-Centered Design for the
EHR
No culture of human-centered design in Health IT
Design without a strategic understanding of what
metrics are needed to improve care
Build without enough input from users
Deploy without iteration and testing
Physician Resentment/Anger
@joyclee

54. Clinical EHR: A combination of
Microsoft Word and Pinterest
@joyclee
Clinicians are inputting
data in unstructured format
in the notes
Data is being lost and/or underutilized
Patient paper questionnaires and
the diabetes data are
scanned to PDF

55. @joyclee
No data
Trapped data
Isolated data
Infrequent data
Confusing data
Suspicious data
Garbage data
PDFMediaTab

56. How do we address
these Diabetes Data
Barriers?
@joyclee

57. @joyclee
PDFMediaTab

58. @joyclee
MediaTab

59. @joycleeKumar RB et al J Am Med Inform Assoc. 2016 May;23(3):532-7.

60. @joyclee
MediaTab

61. Patients, caregivers, clinicians and
researchers work together to choose
care based on best evidence; together
they drive discovery as natural
outgrowth of patient care; and ensure
innovation, quality, safety and value, all
in real-time.
- C3N Project
@joyclee
“

62. Design for Users
Quality
Improvement
Implementation,
Sustainability,
Outcomes
Human-centered
Design
Data Outcomes
Health IT
@joyclee

63. Aim: To decrease the % of
the population with
HbA1c ≥ 9% and increase
the % of the population
with ≥ 0.5% HbA1c
interval improvement
Preference driven
treatment and
effective self-
management
Enhanced registry population
management & Pre-visit planning
Peer/community support
Education/training to support
technology use and patient
viewing and problem solving with
blood glucose data between visits
Interventions/toolkits for
addressing barriers to adherence
Efficient use of
technology and data
to support care
Access to care and
regular follow-up
Screening for depressionPsychosocial
Support
Shared decision making
Partnership between
engaged patients
and the care team
Effective use of EHR by diabetes
team for population management
• % of pts. testing ≥4 times/day
or using CGM (6/7 days/week)
• % of pts. giving 3 or more
short-acting boluses/day
• % of pts reviewing data
between visits
• % pts setting, documenting,
and reviewing goals
• % completed pre-visit planning
• % with ≥ 4 visits per year
• % of pts with annual
CDE/RD/SW visit
• % of pts on case mgmt.
pathway
• % pts screened for depression
Developing a Clear Measurable Aim and
a Theory of Change Care Process Measures
@joyclee

64. Local Infrastructure to support an LHS
Team (Director, Associate Director, Patient Partner/Advisor, Project
Manager/Analyst)
Patient Engagement (Patient Advisor/Advisory Board, Website/Newsletter)
QI interventions (Depression Screening, High-risk Patient Recall, Portal
Onboarding, Data Engagement Curriculum)
Improving Data/Technology Systems
@joyclee

65. @joyclee

66. @joyclee

67. @joyclee

68. @joyclee
“No one is going to use that tool
if you can’t BOLD the text!”
Rogue commas
Tool for Data Input
Insulin sensitivity
12AM 90
2:30 AM 110
4 AM 230*
10 AM 160
Patient Instructions
Insulin sensitivity 12AM
12AM 90
2:30 AM 110
4 AM 23, 0*
10 AM 160

69. @joyclee
MediaTab
Outcomes that Matter
Tools for Structured Data Collection
Patient Reported Outcomes
(Portal Questionnaires)
Clinical Interface Redesign
Tools for Population Management

70. How might our
machine learning
colleagues help?
@joyclee

71. Understand how, when, and what data
is desired by different stakeholders to
solve the right problems
Think ecosystems not apps
@joyclee

72. Population Level Monitoring
Prioritization of high-risk
patients
Tools for Seamless
Data Acquisition
Data Pattern Review
Documentation
Communication

73. Watch patient experts solve problems
@joyclee

74. Who’s the real
medical expert?
@joyclee

76. @joyclee

77. @joyclee

78. @joyclee

79. 21. France
22. Netherlands
23. New Zealand
24. Mexico
25. Croatia
26. South Africa
27. Israel
28. Japan
29. Switzerland
30. Hungary
31. Belarus
29,000+
Int’l
11. Australia
12. Portugal
13. Denmark
14. Germany/Austria
15. Turkey
16. Russia
17. Romania
18. Korea
19. Ireland
20. Brasil
55,000+
worldwide
1. Italy
2. UK
3. Spain
4. Sweden
5. Bulgaria
6. Norway
7. Poland
8. Czech+Slovakia
9. Canada
10. Finland

80. Software Innovations
@joyclee

81. Hardware Innovations
@joyclee

82. How do you get your CGM in the Cloud?
@joyclee

83. “The DIY Artificial Pancreas”
Loop/Loopkit OpenAPS AndroidAPS
@joyclee

84. Interoperability and choice matter
Reduce the mental burden
Proactive not reactive
Be transparent
What should I consider and why?
Give access to all of the data
@joyclee

85. The machine is not an end. An
airplane is not an end: it is a tool.
Tools are created to allow you to
reach greater goals, and machines
should not distract from this pursuit.
In fact, you should barely notice that
they’re there.
@joyclee
“

86. Jacob Dwyer, Ashley Garrity, Valeria Gavrila, Emily Hirschfeld
Dorene Markel, Ram Menon, Amy Ohmer, Lilia Verchichina, Michelle Wichorek,
Pediatric Diabetes Team, The Nightscout Foundation, T1D Exchange
Acknowledgements
@joyclee
Joyce Lee, MD, MPH
doctorasdesigner.com
joyclee@med.umich.edu
http://www.nightscout.info/
https://www.nightscoutfoundation.org/contribute-data/
AHRQ, R21HS023865 and PCORI Engagement Award (1442-UMich)