A joint presentation on Real People, Real Data at the 2016 International Forum on Quality and Safety in Healthcare in Gothenburg, Sweden. Presented by Leanne Wells of the Consumers Health Forum of Australia; Sam Vaillancourt of St. Michael’s Hospital, Toronto, Canada, and; Dr Paresh Dawda of the Australian National University.

1. Patient Reported Outcomes to
Accelerate Change
Leanne Wells (@LeanneWell63)
Sam Vaillancourt (@VaillancourtSam)
Paresh Dawda (@pareshdawda)

2. Session Outline
Healthcare systems are striving to achieve value and
patient-reported outcome measures (PROMs) hold the
promise of focusing quality improvement on what matters
most to patients. Speakers will share the rich experience
of developing PROMs for outpatient emergency care in
Canada with the Australia experience of using a powerful
tool, ‘real people, real data’, designed to capture people’s
stories about their ‘whole of life’ and ‘whole of system’
experiences of health, healthcare and health outcomes.
• Discuss in which circumstances patient-reported outcomes may be useful
and the process of developing reliable and valid PROMs
• Understand the use of the ‘real people, real data’ tool to define the
patient’s perspective, outcomes and value.
• Introduction and overview
– Leanne and Sam
• The case for patient engagement and narratives
– Leanne
• Real People Real Data – a toolkit
– Paresh
• PROM in ED
– Sam
• Key Points
– Leanne

3. Tomorrow’s health care systems
Consumer
transformation
Healthcare
delivery
transformation
Clinical
transformation
Commercial
transformation
Pace of
change

4. What is value?
• Quadruple aims
– Better health outcomes
– Better experience of care
– Better value
– Better supported workforce
• Patients as partners in care
• Consumers as co-creators of value

5. Patient reported
measures
Satisfaction surveys Satisfaction
PREM Experience of care

6. Patient reported
measures
Satisfaction surveys Satisfaction
PREM Experience of care
PROM
Symptoms
Functional status
Health-related quality of life
Other related care outcomes

7. PROMs and PREMs

8. PROMs and PREMs

9. What matters to them?
• A fragmented system and providers working in
isolation not as a team
• Uncoordinated care
• Difficulty finding services
• Service duplication, absent or delayed services
• Low uptake of eHealth and other health technology
• Access problems due to cost, transport,
language, mobility and remoteness
• Feelings of disempowerment

10. What matters to them?

11. • Make life easier, more
convenient for ME
• Let ME take ownership
• Empower ME
• Include and respect ME
in the relationship
• Keep ME informed
• Enable transparent
access to MY info
• Give ME the best care
you can
• Reduce MY costs

12. Patient narrative benefits
• Overcome limitations of traditional methods
• Whole of life and system insights
• Patient life journey approach: track a person’s
ideal health pathway
• What happens versus why
• Quality and safety compliance + innovation
• Effective self management of health
• Overcome risk-averse cultures
Ref: Literature and Practice Review. https://www.chf.org.au/real-people-real-data-
keydocs.php

13. Benefits of patient narratives
Consumer
•cathartic
•affirming
•empowering
Health services
•tool for consumer centeredness
•assist with regulatory compliance
•move from compliance to commitment
in improving outcomes
System
•equity
•system performance
•whole of life
•response to emotional experience
•answer why
•refocus resource allocation

14. ‘Seven Benefits’ framework
• Richer insight
• Potential solutions
• Changing relationships
• Individual benefits
• Better quality decisions
• Changing practice
• Benefits beyond the project
David Gilbert, inhealth associates, UK,
Futurepatients blog, 2015

15. Towards an analysis framework
individual health experiences
expectations
Family/home situation and
support
Work/employment; social
inclusion, community activities
Medical health professionals
Medical procedures,
treatments/devices, medications
Medical & health services
Health system policy & funding;
social determinants

16. REAL PEOPLE, REAL DATA
Using patient stories to shape, design and improve care

17. Exercise
• Spend 3-4 minutes speak to your neighbour
about a story in relation to healthcare
experienced by you or a family member or a
friend.
• Think about:
– How would you describe the relationship between the
different services/professionals that treat your
condition? Who makes decisions about your care and
treatment?
– Did you know what to expect before you had the
treatment/procedure? Were there any surprises?
– Is there anything that’s worked especially well in the
care you’ve received? Is there anything that hasn’t
worked well?

18. Feedback

19. Development
Evaluation
Piloting
4 sites/services
Literature Review
Patient stories Analysis

20. The process
Capture the
narrative
Stages of life
journey/health
process
Analysis to
define themes

23. Diagnosed with breast
cancer.
• Quick access
• Results delayed
• Treatment options

24. • Surgery
• breast cancer on other
side
• Second surgery;
dishcharge issues

25. • Chemotherapy
• Tolerated well
• Side effects but managed
& were explained
• Infected line - sepsis

26. • Relapse
• Quick access, again
delayed
results/treatment plan
• Moved areas

27. • New specialist
• Chemotherapy;
tolerated well but left
her weaker

28. • Spread to further LN; local skin
involvement
• Radiotherapy/Chemotherapy –
weakness; long waits
• needs more help self care; my father now
finding it difficult to coordinate

29. • Further radiotherapy & chemo
• Pneumonia (PCP) admission – serious
• Very frail
• Comes home; falls; admitted. Liver
involvement, transferred to hospice and dies a
few days later on her 71 birthday.

30. Prevention
Change in health
Seeking assistance
Diagnosis
Treatment
Living with a health issue
Recovery
End of life
Access, equity and affordability
Information and understanding
Informed consent (including informed financial consent)
Appropriate care
Respectful care
Whole of person care
Coordinated care and supported transitions
Safety & quality
Control & choice
Social, economic and community participation
Carers & support
Stage of life journey Analysis to define theme

31. Analysis

32. The Health Experience Wheel

33. • A major strength (or value-add) of the
RPRD tool is the consumer story wheel.
This story wheel for presenting the
consumer narrative is the first of its kind
in the world.

34. Practical Application
• “To me it just seemed really rigorous, really
robust, and importantly doing justice to a story.
But it’s not just you telling me some casual story
and then me telling the decision makers. It’s
actually presented as something that fits into their
evidence based process”
• “You don’t need many of those to get a picture,
what the key issues are in the organisation or in a
service. Because they are so powerful in terms of
how they communicate the issues”

35. Applicability
• To determine whether to
introduce new blood test or
not
• Alzheimer's Australia
(http://ihic.improve.org.au/wp-
content/uploads/2015/11/C3_BENNETT.pdf)
• Healthcare Consumers
Association
• Healthcare Services
• Boards
• Frontline

36. https://expwheel.chf.org.au
http://ourhealth.org.au

37. PROM
Data that
speaks for patients
Samuel Vaillancourt
Emergency physician | Associate Scientist
St. Michael’s Hospital
University of Toronto

38. 1. What are Patient reported Measures (PROMs)
2. The Science of PROM
3. PROM-ED for Emergency Department care
Outline

39. Our shared purpose?
• Satisfaction
• Experience
• Outcome

40. -1-
What are Patient reported
Measures (PROMs)

41. Patient-Reported
Outcome Measures
• Not actually outcome
• Well developed in clinical trials. 25% of US drug
labels include patient reported outcome.
• Rapid growth in routine care

42. –Avedis Donabedian
“Outcomes remain the ultimate validators of the
effectiveness and quality of medical care.”
Donabedian, A. (1966). Evaluating the quality of medical care. The Milbank Memorial Fund Quarterly, 44(3),

44. Types of PROMs
• Generic e.g. SF-36 EQ-5D
• Health-related quality of life
• Specific e.g. Oxford Hip
• Condition
• Anatomy
• Care setting

45. -2-
The Science of
PROMs

46. National Quality Forum. Patient Reported Outcomes (PROs) in Performance Measurement. Washington, DC:
National Quality Forum; 2013
Concept
e.g. Person with clinical
depression
PRO
patient-reported outcome
Feeling depressed
PROM
patient-reported outcome
measure
PHQ-9
PRO-PM
patient-reported outcome
performance measure
% patients score > 9
by 3 weeks

47. Potential
• Tracking outcomes
• Assessing symptom severity
• Assisting treatment decision or interventions
• Monitoring general health

48. Using patient reported
outcomes
• Identify issue and population of
interest
• Identify domains of importance
to patients
1. Patient
Reported
Outcome
• Identify existing PROMs
• Test for reliability, validity,
responsiveness
• Test feasibility of use
2. Patient
Reported
Outcome
Measure
• Aggregate PROM data,
benchmark
• Evaluate threats to validity.
E.g. exclusions, missing data,
poor response rate
3. Patient
Reported
Outcome
Performance
Measure
National Quality Forum. Patient Reported Outcomes (PROs) in Performance Measurement.
Washington, DC: National Quality Forum; 2013

49. Define PRO
Define Target Measurement Need
1.Concept: _______________
2.Population: _____________
3.Purpose: ☐ one point in time ☐ change over time ☐ predicting future state

50. Defining PRO
• Review of the literature
• Expert opinion
• Patient interviews
• Patient focus groups

51. Conceptual framework
PRO
Depression
Physical
Functioning
Social
Functioning
Psychological
functioning
Employment
Friends and family
Exercising
concentration
mood

58. -3-
PROM-ED for
Emergency
Department care

62. Exercise
At your table, share an instance when you or a
relative sought care in an Emergency department
1. What were you trying to achieve through ED care?
2. What was important about your experience?
3. Are these two similar or different?

63. Methods
• Patients recruited at their ED visit
• Purposeful sampling
• 45 In-depth interviews 3 to 9 days after ED care

64. Figure 2. Conceptual Model of Patients Conception of ED Care Outcomes
Patient-reported
outcomeofEDcare
Understanding
Explanationforsymptoms(diagnosis)
Implications
Expectedtrajectory(prognosis)
Worry&Distress
Reassurance
Fear alleviated
Feeling of control
SymptomRelief
Direct suffering
Impactonfunction
Having aPlan
Howtoresolveissue
Howtoimprovesymptoms
Howtocontinuediagnosis

65. Key Points
• Patient reported information regarding experience
and outcome is essential.
• These should take many forms, from narratives to
data.
• Doing it right requires critical deliberate
development involving several stakeholders.
• Enabler for improvement

66. Adaptive change
“Technical changes are those with well defined
problems, where a clear solution can be found and
the implementation path is clear...
Adaptive changes are characterised by situations
where the challenge is complex and to solve it
requires transforming long-standing habits
….new ways of thinking and
relationships….The development of new models
of care and many challenges the local systems are
being asked to plan for are in the domain of
adaptive change….”
Nigel Edwards, , 11 March 2016

67. Contact details
• Leanne Wells
– L.Wells@chf.org.au
• Sam Vaillancourt
– sam.vaillancourt@utoronto.ca
• Paresh Dawda
– dr.paresh.dawda@gmail.com