PCORI at Academy Health

K ATHE RI NE B . B E VAN S
AMY K RATC HMAN
C HRI STOP HE R B . FORRE ST
C HI LDRE N’S HOSP I TAL OF P HI LADE LP HI A
UN I VE RSI TY OF P E N N SY LVAN I A
SUCCESSES AND CHALLENGES IN ENHANCING
PATIENT ENGAGEMENT
DEVELOPMENT OF METHODS FOR IDENTIFYING CHILD
AND PARENT HEALTH OUTCOME PRIORITIES

ADVANCES IN PATIENT-REPORTED OUTCOME
MEASUREMENT AND ADOPTION
• NIH Initiatives: Patient-Reported Outcome (PRO)
Measurement Information System® (PROMIS), Neuro-
QoL, NIH Toolbox
• Provides qualitative and quantitative standards for
demonstrating the reliability, validity, sensitivity, and efficiency
of PRO measurement tools.
• Food and Drug Administration guidance
• Integration into EPIC
• Increasing adoption in clinical research and practice

A MISSING PIECE…
• Usually, outcomes are selected without sufficient
patient input
• Precludes assessment of the full spectrum of patient concerns
• May lead to identification of “best practice” interventions that
have little or no effect on the outcomes that matter most to
children and parents
• Core sets = the “minimal standards” for the
assessment and reporting of outcomes in clinical
studies and encounters
• Usually comprised of outcomes identified by researchers and
clinicians
• Some attempts to obtain patient input through qualitative
means (e.g., focus groups)

CHILDREN WITH AUTISM SPECTRUM DISORDERS:
COMMONLY EVALUATED PRO’S VS. PARENT-
IDENTIFIED OUTCOME PRIORITIES
Behavioral and
emotional inhibition
and control
Sensory regulation
Restricted interests,
flexibility, and
adaptation
Stereotyped
behavior
Energy level
Externalizing
behavior
problems
Interpersonal
communication
Anxiety
PRO Parent-Identified

PURPOSE OF PCORI PILOT PROJECT
• To develop and evaluate novel methods for eliciting
and prioritizing children’s health outcomes that are:
• Comprehensible to both patients and clinicians
• Measureable
• Ready for integration into comparative effectiveness research
• Focused on six of the most common childhood
conditions: ASD, ADHD, epilepsy, JIA, asthma, obesity

OUR TEAM
Patient Advisors (6)
Scientific Advisors (3)
Researchers
Bevans +3
Patient-Scientists
Kratchman +3
Harmonization
Concept
Validation
Patient
Understanding
Prioritization
CORES: Co-directed by a researcher
and a patient-scientist
Patient Advocacy
and Support Groups

CHALLENGES AND LESSONS LEARNED:
DEVELOPING THE TEAM
• Inclusion of “representative” patients
• Recognize and respect varying levels of patient
engagement
• Related to patients’ readiness to engage as a patient-scientist
• Reasons for readiness = personal journey, experience,
confidence…
• Researchers’ readiness to adopt the PCOR model
• Power and team structure
• Communication: Learning the language vs. using a
translator
• Consider researchers’ and stakeholders’ unique and
complementary expertise in assigning roles and
responsibilities

AIM 1: DEVELOP A TYPOLOGY OF HEALTH
OUTCOMES
ICF-CY,
PRO
concepts,
NCS
Intervention Research
Outcomes (Systematic
Literature Review)
Clinician-Identified
Outcomes
Child- and Parent-
Identified Outcomes
Focus on six common childhood conditions: ASD, ADHD,
epilepsy, asthma, JIA, obesity

AIM 1: DEVELOP A TYPOLOGY OF HEALTH
OUTCOMES
ICF-CY,
PRO
concepts,
NCS
Literature Review)
Outcomes
Child- and Parent-
Identified Outcomes
Researcher-
Driven

Develop Controlled
Vocabulary
Condition-Specific AND
Measurement AND Self/Proxy-
Report
Limited to Children
Execute Search
PubMed
3189 articles
Identify Unique Article
Remove Duplicates
2865 articles
Abstraction of Articles
In-Scope
201 ASD articles (19%)
309 ADHD articles (39%)
292 epilepsy articles (29%)
Identification of Tools
40 ASD tools
36 ADHD tools
28 epilepsy tools
Obtain Tools
32 ASD tools
31 ADHD tools
24 epilepsy tools
Examine and Classify Items until
Concept Saturation is Achieved
1592 ASD items
698 ADHD items
Epilepsy items (TBD)
IDENTIFYING COMMONLY ASSESSED OUTCOMES
SYSTEMATIC LITERATURE REVIEW

TYPOLOGY MAPPING:
VINELAND SOCIALIZATION
SCALE
Most commonly assessed ICF-CY categories:
b126: temperament and personality functions
d125: disposition and interpersonal functions
b125: emotional functions
d710: basic interpersonal interactions
d720: complex interpersonal interactions
d750: informal social relationships
d330: speaking
d880: engagement in play
d920: recreation and leisure

AIM 1: DEVELOP A TYPOLOGY OF
HEALTH OUTCOMES
ICF-CY,
PRO
concepts,
NCS
Literature Review)
Outcomes
Child- and Parent-
Identified Outcomes
Patient-Scientist
Driven

ASSESSING CHILD AND PARENT
PERSPECTIVES
What problems
would you like
your child’s
treatment to
address?
How would you
like your child’s
treatment to
improve his/her
life?
A lot of research
involving children with
asthma focus on
symptoms like difficulty
breathing, but asthma
impacts kids in other
ways too. How does
your child’s asthma
impact his/her life?
How would you like
that to change?
(Probe specific life
domains: at school, with
friends, how he/she
communicates, etc.)
• Preference for
positive focus
• “Treatment”
facilitates focus
on symptomatic
outcomes
• Need to be
specific about
condition
• Consider impact
on family, not just
the child
Participants: Youth aged 12+ with six conditions and their parents;
two in-person and two online data-collection modalities

AIM 1 PRODUCT
Outcome typology derived from patients (parents
and youth), clinicians, and a review of the
intervention research

CHALLENGES AND LESSONS LEARNED: AIM 1
• Empower patient-scientists to lead development of
methodology
• Methods for accessing
and engaging patient
partners to elicit patient-
centered outcome
priorities
Data Quality
Access Feasibility
Acceptability
Network
Barriers:
Geography
Sample Bias Cost

AIM 2: ENSURE THE COMPREHENSIBILITY AND
CONTENT VALIDITY OF THE TYPOLOGY
e-Delphi with clinicians
Round 1: Open-ended elicitation
of outcome concept definitions
for clinicians and patients
Rounds 2 and 3:
Rate each outcome definition:
• Clinician understanding
• Patient understanding
Suggest revisions
2x
Cognitive Interviews with
youth and parents
Development of clinician
and patient outcome
definitions
Revisions
Test patient-centered outcome
definitions using a cognitive
interview approach
Iteratively refine
and re-test until
definitions are well-
understood
Outcomes defined in BOTH clinician-
oriented and lay language

AIM 2 PRODUCT
Typology with outcomes defined in clinician-oriented
and lay language.
Uses:
• To translate and standardize the dialects that health professionals
and patients use to describe health experiences
• As a substrate for the outcome prioritization process

CHALLENGES AND LESSONS LEARNED: AIM 2
• Communication: Learning the language vs. using a
translator
• Use the expertise of patient-scientists
• Capitalize on clinicians’ experience in communicating with
patients
• Commitment to developing a replicable and
scalable approach to identifying patient-centered
outcomes (rather than a product)

AIM 3: DEVELOP AND TEST METHODS FOR
OBTAINING PATIENT-OUTCOME PRIORITIES
• Test two technology-enabled methods for prioritizing
patient outcomes
• Method 1: Individually administered
• Participants (youth with ASD and parents of children with ASD)
are asked to choose between two outcomes (with patient
definitions).
• Outcome ratings are determined by analyzing results of the
pair-wise comparisons using analytic hierarchy processes.
• Method 2: Administered in groups of similar
participants
• Participants rate the importance of each outcome using an
audience response system.
• After each question, response frequencies are shown in real-
time to the group to facilitate discussion.

AIM 3 PRODUCT
A scalable approach to identifying patient-prioritized
outcomes
Uses:
• To establish “patient-centered” core sets for use in clinical
research involving specific sub-groups (e.g., diagnostic,
capacity, age)
• For further testing in clinical settings as a means for identifying
individual patient’s outcome priorities

AIM 3: ANTICIPATED CHALLENGES AND
FUTURE DIRECTIONS
• Selecting an outcome prioritization approach that is
understandable, is feasible, and yields meaningful
results
• Promote approaches for developing “patient-
centered core sets”
• Patient prioritization of outcomes included in PRO
development recommendations/standards
• Sustain the patient-scientist network
• Involve patient-scientists in interpretation of results and
dissemination efforts
• Strengthen and deepen engagement

Contact:
Katherine Bevans, PhD
Assistant Professor of
Pediatrics
CHOP/Penn Medicine
bevans@email.chop.edu
267-426-2967

PCORI PILOT PROJECTS
Aims
1. To develop a typology of health outcomes that is
relevant to the experiences and preferences of
children.
2. To ensure the comprehensibility of the health
outcome typology for children and their parents.
3. To develop and test the feasibility and
practicality of methods that use the typology to
prioritize health outcomes from the perspectives
of children and their families, using Autism
Spectrum Disorder as a proof-of-principle
condition.

Patient and Clinician
Perceptions of Engagement in
Research
1
Laura Forsythe, PhD, MPH
AcademyHealth Annual Research Meeting
June 25, 2013

 Improved research recruitment and retention rates
(Edwards et al. 2011)
 Enhanced trust between researchers and participants
(Decker et al. 2010; Edwards et al. 2011; Staniszewska et al . 2007)
 Improved content and construct validity of measures
(Cashman et al. 2008; Cotterell 2008)
 Improved patient understanding of results
(Chalmers 1995; McCauley et al. 2001; Doyle 2010)
 Increased relevance of research results to patients
(summarized in Nass et al. 2012)
Patient Engagement in Research

Objectives
Describe and compare patient and clinician
attitudes about engagement in research
Identify perceived barriers to and facilitators of
engagement in research
3

Methods: Survey Development
5
Identify Existing
Survey Items
Develop New
Survey Items
• Perceived value of
engagement
• Interest in engagement
• Barriers and facilitators
for engagement
Partner with Patients
and Clinicians for
Feedback
• Survey concepts
• Item wording
• Survey layout
• Dissemination

Methods: Crowdsourced Survey
Recruitment from existing opt-in panels based on
pre-supplied profile information
Web based survey
Rapid data collection
Limited generalizability
6

Methods: Item Format Example
7

Respondents: Patients (N=900)
8
80%
20%
Disease group
Chronic disease patients
Rare disease patients
89%
11%
Primary Language
English Spanish

Respondents: Primary Care Clinicians
(N=750)
9
53%
27%
12%
8%
Type of Provider
Physicians
Nurse Practitioners
Nurses
Physician Assistants
7%
23%
35%
25%
10%
Years in Practice
< 3 Years
3 to 9 Years
10 to 19 Years
20 to 29 Years
>30 Years

Perceived Value of Research that Measures
Things Patients Care About
11
87% 87%
0%
20%
40%
60%
80%
100%
Patients Clinicians
%Very/ModeratelyImportant
Survey Group
p>0.05

72%
77%
0%
20%
40%
60%
80%
100%
Patients Clinicians
%Strongly/SomewhatAgree
Survey Group
*p <0.05
Perceptions that Research Helps Patients
Make Better Treatment Decisions
12

Perceived Value of Engaging Patients in
Research
13
86%
72%
83%
0%
20%
40%
60%
80%
100%
CliniciansCliniciansPatients
Survey Group
*p<0.001
“Patients working directly with
researchers can improve the
value of medical research”

Perceived Value of Engaging Clinicians in
Research
14
86%
72%
83%
0%
20%
40%
60%
80%
100%
CliniciansCliniciansPatients
Survey Group
*p<0.001
“Patients working directly with
“Providers working directly with

Interest in Engaging in Research
15
66%
55%
0%
20%
40%
60%
80%
100%
Patients Clinicians
Survey Group
*p<0.001

Barriers and Facilitators of Engagement
16
Barriers Facilitators
Patients
• Lack of time (43%)
• Concerns about privacy
(36%)
• Work, school or caregiving
commitments (33%)
• Helping others with their medical condition
(68%)
• Learning about their health (63%)
• Helping the next generation (57%)
• Getting paid (56%)
• Making research more meaningful to
patients (49%)
Clinicians
• Lack of time (79%)
• Lack of payment (47%)
• Lack of research training
(35%)
• Helping patients receive better care (79%)
• Getting paid (78%)
• Contributing to scientific knowledge (61%)
• Making research more meaningful for
patients (61%)
• Improving professional satisfaction (52%)
• Helping researchers decide what to study
(43%)

Strengths and Limitations
Strengths
 Exploration of understudied topic areas
 Inclusiveness of understudied populations: Spanish
speakers, rare-disease patients
Limitations
 Generalizability
 Self-reported data
 New survey items testing complex constructs
17

Conclusions
Most patients and clinicians agreed that
engagement can improve the value of health
research.
Many patients and clinicians reported interest in
engaging in research themselves.
Strategies to facilitate both patient and clinician
engagement:
 Establish link between engagement and patient care.
 Provide financial compensation.
 Minimize time burden.
18

Implications
Find ways to help those who are interested in
engaging in research to be involved in meaningful
ways.
Develop evidence on the value of engagement in
research, particularly for improving patient
outcomes.
Address barriers to engaging patients and primary
care clinicians in research.
19

Next Steps
20
Foundational Elements
• Awareness of methods for PCOR
• Valuing patient perspective
• Interest in PCOR
• Ways for patients and researchers to partner
• Resources and infrastructure
• Policies/governance

Thank you!
Acknowledgements
 Patient, caregiver, and clinician partners
 Collaborators: Lori Frank, Kara Odom Walker, Diane
Hayes, Sue Levine, Ayodola Anise, Natalie Wegener,
Gail Hunt, Anne Beal, Harlan Weisman, Freda Lewis Hall
Laura Forsythe, PhD, MPH
lforsythe@pcori.org
www.pcori.org
21

Developing Conceptual
Frameworks to Inform and
Guide PCOR
25 June 2013

Understanding
the choices
patients face
Aligning research questions
and methods with
patient needs
Providing patients and
providers with information
for better decisions
Patient Engagement
Patient-Driven
Research Dissemination
Taking Patient-Centeredness Seriously

Research engagement
 Who determines
 The research questions?
 The database to use?
 The variables to examine outcomes,
comparators, and covariates?
 The analytic methods?
 Who is on the research team?
 Patients or patient advocates?
 Other stakeholders?
 What will the research team do with the information?
 How will results apply to health decisions?
 Is there a plan for interaction between researchers and the
community?

Frameworks for PCOR
Identify PCOR concepts and relationships between
them
Identify key components of PCOR and role of
engagement
Theoretical and evaluative frameworks:
 One conceptual model for PCOR
 A distinct measurement model

PCORI Conceptual Model of PCOR
Process for development informed by:
 Evaluative framework of research engagement
(AcademyHealth)
 Review of PCOR literature
Refined and vetted through:
 Discussion with AcademyHealth
 Input from PCORI’s Patient Engagement Advisory Panel

PCOR conceptual model
The following key changes were made based on
discussion with the patient engagement advisory
panel:
 TBD
 TBD
Final structure: PCOR conceptual model
 Use as basis for deriving testable measurement models.

Foundational Elements
• Awareness of methods for PCOR
• Valuing of the patient perspective
• Interest in PCOR
• Ways for patients and researchers to partner
• Resources and infrastructure
• Policies/governance
Actions
• Initiate and maintain partnerships
between researchers and stakeholders
• Facilitate cross-communication among
research stakeholders
• Capture, use, and optimize patient
perspective across phases of research
• Ensure meaningful influence on
research
• Train for partnering
• Share and use learnings
Outcomes
PCOR Principles
Trust, Co-learning, Transparency, Respect
• Culture of patient-centeredness
in research
• Meaningful & effective
partnerships
• Optimal health outcomes
• Research relevant to
patients/other stakeholders
• Use of research results in
health decisions
• Quality health decisions
• Satisfaction with health care
experiences
Conceptual Model for PCOR
Outcomes
Near-term
Long-term
Intermedia

AcademyHealth and PCORI Partnership
Contract awarded in 2012 to:
 Conduct literature scan on stakeholder engagement in
research (manuscript in progress)
 Develop framework for patient engagement in research
 Establish learning network for 50 PCORI pilot projects
 Monitor progress and experiences of PCORI pilot
projects in key areas
8www.pcori.org

PCORI Pilot Projects Program
50 projects in 24 states and Washington, DC
$31 million (over two years)
www.pcori.org 9

AcademyHealth Approach
Framework for patient and consumer engagement
in research:
 Literature scan
 Initial framework draft (input from PCORI staff)
 Three iterations of input from Consumer Patient
Researcher Roundtable
 Revision and augmentation ongoing

Evaluative Framework for Patient
Engagement in Research
Key Attributes
 Intended for use as evaluative tool
 Based on Donabedian model
 Agnostic with respect to entity “sponsoring” research
effort
 Inclusive definition of the term “patient”
 Acknowledges how much remains to be learned about
how to conduct PCOR

Evaluative Framework for Patient
Key Components:
 Structure: refers to existence of assets, policies, and
other infrastructure deemed necessary for conduct of
PCOR
 Process: examines how patient engagement is
incorporated or integrated into research process
 Outcomes: near and longer-term end points of patient
engagement in research (such as improvement in
research design, health outcomes)

Engagement Infrastructure
Engagement
infrastructure
Patient/researcher
matching and
outreach
Patient-to-
researcher mapping
and selection
Research stage(s)
of engagement
Continuity and
frequency
Structure Process

Increased quality
and/or relevance of
research
More informed
decision making and
uptake of research
Improvements to
dissemination and
access to research
Policy
deliberations/changes
Improvements in
patient activation and
empowerment
Improvements in
health outcomes and
health status
Outcomes
Attitudes,
perceptions, and
activation
Modifications to
research process
Interim
Outcomes
Longer-term
Outcomes

Next Steps
Anticipated:
 Still a work in progress, so welcome questions,
suggestions, and other feedback
 Developing manuscript for peer review
Hoped:
 Would like the framework to be used as a guidance tool
for those engaged in PCOR
• Not a prescriptive “how to”

Thank You!
Consumer Patient Researcher Roundtable
Contributors: Bryan Dowd, Jason Goldwater,
Mark Gorman, Alice Leiter, Musa Mayer, Eva
Powell, John Santa, Shoshanna Sofaer, and Mike
Stoto
AcademyHealth and PCORI Staff: Rochelle Bent,
Laura Esmail, Laura Forsythe, Lori Frank, Emily
Moore, Raj Sabharwal, Veronica Thomas, Rachel
Witsaman, and Natalie Wegener

Improving the Impact of Patient-Engaged
Research: Recommendations for
Evaluation
Lori Frank, PhD and Sue Sheridan, MIM, MBA
AcademyHealth Annual Research Meeting
June 25, 2013

October 2012
Transforming Patient-Centered Research:
Building Partnerships and Promising Models
Workshop purpose: Begin a dialogue on how to conduct
patient-centered research.
Input received was used by PCORI staff to develop policy
and programmatic recommendations.
2

Workshop Structure
The two-day event included 142 representatives from a
range of communities.
 75% of participants were patient representatives.
 Other participants: clinicians, providers, researchers
Small group brainstorming sessions
3

Breakout Session Topics
4
Identifying and Selecting
Research Questions
How should PCORI identify and select specific research
questions that are patient-centered for funding ?
Reviewing Research Proposals
for Funding
How can PCORI effectively engage and use the real-world
experience of patients to help evaluate research
proposals we receive?
Matching Patients and
Stakeholders with Researchers
How can PCORI connect patients and stakeholders with
researchers for collaborative work that ensures studies
reflect patient perspectives?
Disseminating Research to the
Community
How do we ensure that patients and those who care for
them can access and use PCORI’s research to make more-
informed decisions?
Evaluating PCORI’s Patient and
Stakeholder Engagement
Programs
How can PCORI measure the effectiveness of its programs
to involve patients and stakeholders throughout its work?

Evaluating Engagement Session:
Questions for Consideration
Participants were asked to consider:
 How can PCORI best measure the effectiveness of
patient and stakeholder engagement in research?
 What novel methods can patients and patient advocates
propose for evaluation of research engagement that
would capitalize on the growing networks of patients
engaged in research?
5

Evaluating Engagement in Research
6

Nominations for
Principles of Engagement
Transparency Empathy
Infrastructure
change to achieve
parity between
researchers and
patients
Incorporating
learning in an
ongoing way
Aiming for
excellence in
engagement
Evaluating the
meaningfulness,
quality, and impact
of the engagement

Recommendations for Evaluating
• The objective for engaging patients and researchers as partners in research may be study-
specific, but must be pre-defined to permit adequate evaluation of the partnership.
Define Success
• Topic generation will require different forms of engagement than will establishing research
hypotheses or analyzing data and reporting findings.
Determine goals of engagement by research phase
• Use to ensure that all research partners can provide input on the quality and value of the
research engagement, using established metrics based on definitions of success.
Establish feedback channels
• Including compensation for patient time and co-authorship
Address patient and researcher parity
• Assessments should happen before and after engagement.
Assess patient and researcher perceptions of the value and
appropriateness of engagement
8

Items for Evaluating
 Stakeholders: How did engagement in the research change
the project?
 Has your PCORI-funded project been used by other research
groups?
 Was there a successful outcome for your study? Does it relate
to quality of patient engagement?
 Are people voting with their feet?
 Was the research idea collaborative?
 How quickly are results being used in further research and in
clinical care?
Additional metrics should include both subjective and
objective measures of quality of engagement.
9

Process Recommendations for Evaluation
Funding applicants should be asked to build an evaluation
of patient engagement into their proposals for the
conclusion of their studies.
Dissemination metrics: Go beyond peer-review.
Evaluate quality and impact of engagement during and after
the study.
Evaluate diversity of project participants.
Consider linking funding to evaluation results,
10

Recommendations for Strengthening
Incentivize co-learning between researchers and
patient partners.
Organizational structures of health systems and
research organizations should include patient research
experts.
“Seal of Approval” recognition of optimal research
engagement methods
Require it: Evaluate engagement in all PCORI funded
projects.
11

Conclusions
Patients and other stakeholders provided concrete and
feasible recommendations for evaluating quality of
engagement in research.
Metrics for success should be linked to quality of
research and quality of engagement.
Different metrics are needed by research phase.
Organizational structures and funding requirements
can be powerful levers for ensuring engagement
quality.
12

Implications for Policy, Delivery, and
Practice
Meaningfulness and impact of the research should be
measured and link to engagement quality made overt.
Research organizational structures can be modified to
support success.
13

Thank You!
14
Patient, caregiver, and clinician partners
Contact:
Lori Frank, PhD
lfrank@pcori.org
Sue Sheridan, MIM, MBA
ssheridan@pcori.org

Engaging Stakeholders to
Improve Depression
Management in a Tribal
Health System
1
Renee Robinson (PI)
Jennifer Shaw
Vanessa Hiratsuka
Sara Norman
Julia Smith
Denise Dillard
Southcentral Foundation
Helene Starks
University of Washington

Presentation Objectives and
PCORI Pilot Project Overview
• Familiarize audience with…
• Organization, population, and stakeholders
• Depression management at our organization
• Decision-support tools
• Stakeholder engagement
• Approval versus engagement
• Impact of engagement on process/project
• Barriers encountered/proposed solutions
2

Southcentral Foundation (SCF)
• Mission
– Working together with the
Native Community to
achieve wellness through
health and related services
• Vision
– A Native Community that
enjoys physical, mental,
emotional, and spiritual
wellness
3

Population
• SCF provides services to…
– ~60,000 AN/AI people
– Across 107,413 square miles
– 231 federally recognized tribes
– 60 villages
4

Our Stakeholders
• AAIRB
• Tribal Leaders (EC, BOD)
• Customer-owners
• Providers
• Steering Committee
• Leaders
5
Customer-
Owners
ProvidersLeaders

Depression Management
at SCF
• In 2001, SCF integrated
annual depression
screening into all
primary care clinics due
to:
– High prevalence of
symptoms
– Low utilization of
behavioral health
services
6

To make a good healthcare decision,
you need to…
• Clarify the decision…
– What is the decision you
need to make?
– What is your reason for
making this decision?
– Look at timeline for the
decision.
• Explore the decision…
– What healthcare options
are available to you?
– What are positive and
negative effects of
options?
– What is the value of the
positive and negative
effects to you? (Explore
your decision.)
7

Decision-Support Tools
• Interactive tool to aid clinical decision-making
• Describe treatment options, risks, benefits, and efficacy
• Help clarify preferences and values for treatment options
• Communication between customer-owners and providers
• Facilitate improved depression management
– Educational
– Interactive
– Selective
8

Pilot Project
• Background and Rationale…
– Many people don’t pursue depression treatment
• Negative effects of untreated illness (e.g., absenteeism)
– Many people prematurely discontinue treatment
• Wasted resources of healthcare system (e.g., unused Rx)
– Few people prefer holistic depression treatment
• Under-utilized services
9

Project Aims and Overview
10
Aim 1
• Identify stakeholder priorities, preferences,
and needs for depression management
• Key informant interviews with customer-
owners, providers, and leaders
Aim 2
• Develop and evaluate decision-support
tool
• Steering committee guidance and feedback
• Pilot test in one primary-care clinic
Aim 3
• Evaluate health and economic outcomes
• Test refined tool in multiple clinics
• Evaluate disease management, economic, and
health outcomes

Stakeholder Engagement:
Aim 1
• Data Collection
– 38 stakeholder interviews conducted Dec to Feb
• 19 Customer-Owners, 9 Leaders, 10 Providers
– Semi-structured interview guide
• Depression screening and treatment resources
• Decision-tool content, design, and deployment
• Analysis
– Transcripts summarized
– Themes identified
– Synthesized information presented to Steering Committee
• Used to guide tool development
11

Deciding on Drug Therapy
12
I didn't even realize that I was
depressed for a long time. I thought
everyone felt this way; at least,
everyone in my family seemed to. I
probably would have just gone on
like that if my doctor hadn't asked
one day if I had ever thought about
taking an antidepressant. I was
relieved to find out that it isn't
normal to feel like I do and that a lot
of people are helped by medicines. I
know it might take a while to find
the right one, but I'm in no hurry;
I've spent my whole life feeling sad.
I guess I'm just not comfortable with
taking medicine for my depression. I feel
like I ought to be able to manage this on
my own without needing medicine. It
seems too much like taking the easy route.
But maybe I just don't feel bad enough yet.
I recently began going to counseling. I
know that if I took an antidepressant, I
might feel better sooner, but I don't like
the sound of the side effects I could
have. My therapist and I have set some
goals for me to work on, and we agreed
to revisit my decision in three months. I
want to wait and see how the counseling
goes before I take medicine.

Example Screens of First Draft of
Decision-Support Tool
Approval versus Engagement
13

Get the facts
• What is depression?
– A very common, highly treatable,
medical illness that involves the
body, mood, and thoughts. Physical, mental,
emotional, and spiritual well-being.
– It affects how people think about things, feel about
themselves, the way a person functions socially, at
work and in relationships, as well as everyday
activities like eating and sleeping.
– It is more than feeling blue, down in the dumps or sad
about a particular issue or situation.(Link to Table with
Symptoms of Depression, See next slide)
Link to Video about learn more about depression
http://www.youtube.com/watch?v=IeZCmqePLzM&playnext=1&list=PL3E95B34CC364B7D0&feature=results_main
14

What healthcare options are available
to you to manage your depression…
• Lifestyle Changes (Link to page expanding each)
– Healthy eating
– Exercise
– Relationship building
– Stress management
• Behavioral Health Services - Therapy
– Screening
– Psychotherapy
• Medications
• Traditional Healing
• Other

Modifications to
Impact of Engagement on Process
Barrier and Proposed Solutions
16

Stakeholder Recommendations:
Decision Tool Content
• Provide context for the tool
– Who will see the information and how will it be used
• Give people information about depression
– De-stigmatize/normalize (common life experience)
– High level of depression in Alaska due to multiple factors
(seasonal, trauma, situational)
– Different types (situational versus chronic/organic)
• Assess/offer resources
– SCF and non-SCF resources, including Customer-Owner
personal resources (social support)
– Include non-medication resources (dietician, traditional
healing, individual counseling, and groups)
– Explain potential benefits and burdens (side effects, wait
times)
– Follow-up/make multiple contacts
• Use story/culturally consonant communication
– Include Native faces/voices; include testimonials of success
Context
Information
Resources
Story
17

• Adapt for multiple audiences
– iPad generally endorsed for providing screener and general
information, especially with younger people
– Elders and others may prefer personal or written option
• Strategize location and timing
– Could deploy screen in waiting area or provide general info on
closed-circuit TV in waiting area and use iPad for screen in
examination or talking room
• Maximize customer choice
– Make tool voluntary, not mandatory
– Provide staff assistance with tool (e.g., RN or BHC)
– Allow time for customers to explore options, including time in
examination room and in follow-up appointments
• Align with other initiatives
– Minimize duplication and multiple requests for information
Stakeholder Recommendations:
Decision Tool Design and Deployment
Alignment
Audience
Strategy
Choice
18

Factors Related to Workflow
• Administration of tool
– BHC
– PCP
• Role of tool
– Facilitate discussion
– Provide information
– Support Decision
• Time
19

Example Screens of Resultant
Impact of Engagement on Process
20

The Signs and Symptoms that Bother
Me Most Are …
Feel Hopeless
Loss of Interest in
Daily Activities
Weight and/or
Appetite Changes
Sleep
Changes
Anger or
Irritability
Decreased
Energy
Self Loathing
Trouble
Concentrating
Pain
Exit

What can I do about my depression?
Healthy Eating
Exercise
Stress Reduction
Support Groups
Clergy
Counseling
Traditional
Healing
Drugs
Herbal
Remedies
Exit

Engagement Barriers and
Proposed Solutions
• Barriers
– Cultural
– Time
– Communication
– Information
• Solution
– Cultural-adaptation
– Technology/resources
– Decision-support tool
– Decision-support tool
and extended tool
23

Other Example Screen Shots of
Old and New Tools
24

Every day people face complex
health decisions…
However, most healthcare decisions
have no clear best choice.
0%
10%
20%
30%
40%
50%
Beneficial
Outcomes
Probably
Beneficial
Need to
Weigh
Benefits
versus Risk
Probably Not
Beneficial
Insufficient
Evidence of
Usefulness
Healthcare Intervention
Classifications
25

Balance, Health and Wellness
Exit

What causes depression?
Life Experiences Biology
Exit
Money problems/
unemployment
Grief and loss
Violence Drugs/ Alcohol
Abuse
Genetics Neurotransmitters
(brain chemicals)
Hormones Other Medical
Conditions

AcademyHealth ARM
PCORI Engagement Panel
ADDRESSING MENTAL
HEALTH NEEDS OF RURAL
AFRICAN AMERICANS:
LESSONS FROM
ENGAGEMENT

Acknowledgements
Funding:
• UAMS Translational Research Institute (UL1RR029884; KL2RR029883)
• Patient-Centered Outcomes Research Institute (PCORI)

This Presentation Will:
1. Describe the process of engaging stakeholders in our pilot project
2. Identify lessons about engaging stakeholders
3. Briefly describe the current status of the project

The setting: Arkansas Delta
• Primarily rural with agriculture-
based economy
• Characterized by:
• Poor economic conditions
• Higher prevalence of chronic health
conditions
• Increased risk of early mortality
• Poor access to quality health services
• Highest percentage of cities with
predominately African-American
populations

Mental Health in the Delta
• Poorer self-reported mental
health
• Increased levels of frequent mental
distress (BFRSS)
• Poorer mental health outcomes
• Increased burden of disability
• Increased mortality
• Poorer management of chronic physical
health
• Underuse of mental health
services
• Limited access
• Stigma

UAMS Psychiatric Research Institute
• Division of Health Services
Research
• Improve the lives of individuals with
mental health or substance use disorders
• Focus on rural areas
• Focus on partnership with key
stakeholders
What can we do to improve mental
health service use in rural
Arkansas?

LESSONS
FROM
ENGAGEMENT
Decide who you need to partner with:
• What type of expertise do you need to answer your
question?
• Ground
• People affected by the condition
• Grassroots
• People and organizations who are near the ground
• Caregivers, support system, and service
organization
• Grass tops
• Policymakers

The Partners: Tri County Rural Health Network
• Improves the health of individuals
in the Delta
• Uses community health worker model
• Connects community members to health
resources in rural counties
• Works with UAMS College of
Public Health to create “research
friendly communities”
• Connects with community
members and policymakers

Mental Health in the Delta
I’m worried about our minds. About our ability to
cope with life. Especially with our young people. I
used to didn’t hear about depression or suicide.
That was never something you heard about with
our people. Now, I have had to sit with several
families of young Blacks who’ve taken their own
lives. I have people who are depressed in my
congregation. Something has to be done.
- A pastor from the Arkansas Delta

LESSONS
FROM
ENGAGEMENT
Determine if you are ready to engage:
1. Do you have similar values/missions?
• Both focused on improving health in rural areas
2. Do you have the capacity to work
together?
• Tri County experience with research
• DHSR experience with community engagement
3. Will this partnership be mutually
beneficial?

What’s the Solution?
• Focus on improving access to and
use of mental health services
• Adapt community-connector program
• Build on both partners’ expertise
• Start with identifying the mental
health needs of rural African
Americans
• Important to hear the voice of the
community
DHSR’s Idea Tri County’s Idea
Tri County was originally created to provide job
training in the Delta. But the people said, ‘We ain’t
got no jobs here, so we don’t need no job training.
What we do need is better healthcare.’ Let’s hear
the community before deciding where to start.

Our Idea
• Conduct formative research to:
• Better understand the mental health needs of rural African Americans
• Inform the development of culturally appropriate interventions to improve
mental health in rural African American communities

Writing the Application
• Focus Groups
• Researcher identifies stakeholders
• Researcher creates interview guide
• More traditional research method
• Deliberative Democracy
• Allows community to self-identify as
stakeholders
• Community creates the frame
DHSR Methods Tri County’s Methods
The energy is important. Forums create energy
around an idea. What would happen if we
implemented an intervention in an area that has
this energy already buzzing?

Writing the Application
• New Questions:
• What is the best way to gather stakeholder input?
• Does one method lead to more activation?
• Is activation important in intervention implementation?

Specific Aims
• Compare two ways of gathering stakeholder input: deliberative
democracy and focus groups
• Themes
• Empowerment
• Gather information to inform the development of an intervention to
improve mental health in rural African American communities

LESSONS
FROM
ENGAGEMENT
Engagement before Funding
1. Helps decide what are the most
relevant research questions
• Whole idea changed as a result of partner input
2. More responsive questions= Better
applications

Starting the Research
• Community Advisory Board
• One provider
• Two mental health consumers
• One connector
• One college student
• One clergy
• One lay community member
• Advises on consent procedures, measurement tools, recruitment, and
data analysis

Mental Health versus Emotional Wellness
No one is going to talk to you about mental
health. When people hear mental health they
think of crazy. Ain’t nobody going to talk to you
about being crazy. I ain’t crazy. I don’t know how
to help you help crazy people
If you want my expertise, you have to ask me
about things I know about.
CAB member

Mental Health versus Emotional Wellness
• Based on CAB feedback:
• Application written using the medical frame
• Wellness framework is more culturally acceptable
• Interview guides and framing sessions used term “emotional wellness”
instead of “mental health”

LESSONS
FROM
ENGAGEMENT
Respect your partner’s expertise
1. Partner is a part of the research team
• Not just approval
2. Listen and incorporate ideas when
possible
• Using wellness frame versus illness frame
• Changing wording of measurement tools
• Not paying unless they finish
3. Respect builds trust
• Trust builds better partnerships

LESSONS
FROM
ENGAGEMENT
Prepare for some differences of opinions
• Audio recording versus note taking
But remember:
• Your core values; what are you working towards
• Respect each other’s point of view and expertise

Where are we now?
• Completed six focus groups (n=50)
• Faith community
• College students and administrators
• Patients
• Providers
• Currently completing community forums (n=86)
• Four forums completed (two more planned)
• Lay community members
• Service organization leaders
• Political leaders

Where are we now?
• Preliminary analysis
• Stigma and low mental health literacy major barriers to care
• Importance of “reaching people where they are”
• Community-based services versus clinic-based services
• Importance of community support in prevention, treatment, and recovery
• Provide education and support
• Address contextual causes that affect emotional wellness

Conclusions
Engagement can lead to:
1. More culturally acceptable research questions
2. Applications that are more responsive to community needs
When engaging stakeholders in research:
1. Make sure you are ready to engage in research
2. Engage stakeholders before funding
3. Respect your partners’ expertise
4. Prepare for differences of opinions

Any real attempt to address the problems that are plaguing our
community will take everyone coming to the table; the grassroots, the
grass-tops, and all the grass in between. Everyone talks about how
difficult it is to bring everyone together, but if you focus on solving the
problem, the mountains become just bumps in the road.
Mary Olson, Tri County Rural Health

Academic Team
Greer Sullivan, MD MSPH (PI) Mary Olson, D. Min (Community PI)
Geoff Curran, PhD Ann Cheney, PhD
Keneshia Bryant, PhD FNP Christina Reaves, MPH
Naomi Cottoms Elise Allee
Faye Smith

Community Advisory Board
Rev. George Barnes Linda Cole
Melva Trask Gloria Scott
Edlun Marshall Pamela Barnes Earnest
Virgil Ward

PCORI at Academy Health

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a PCORI at Academy Health

Similar a PCORI at Academy Health (20)

Más de Patient-Centered Outcomes Research Institute

Más de Patient-Centered Outcomes Research Institute (20)

Último

Último (20)

PCORI at Academy Health