Retrospective
Mixed Methods Analysis of
Practice Transformation
Michael K Magill, MD
Professor and Chairman
Department of Family and Preventive Medicine
University of Utah School of Medicine and Community Clinics
AHRQ Grants #
HS019136-01 (TPC)
HS20106-01 (ARRA-SSCM)

Interdisciplinary Team
Julie Day, MD
University of Utah Community Clinics
JaeWhan Kim, PhD
School of Medicine, Dept of Family & Preventive Medicine
Annie Sheets Mervis, MSW
University of Utah Community Clinics
Debra L. Scammon, PhD
David Eccles School of Business, Dept of Marketing
Andrada Tomoaia-Cotisel, MPH, MHA
School of Medicine, Dept of Family & Preventive Medicine
Norman J Waitzman, PhD
College of Social and Behavioral Science, Dept of Economics

Visits: 300,000+
Active patients: 157,000
11 Community Clinics

University of Utah Community Clinics
Clinic Year Opened
Total
Providers
Primary Care
Providers
Visits Per
Year (FY09)
Madsen 1975 6 5 18,970
Greenwood 1976 17 10 54,475
Redwood 1985 20 10 93,110
Westridge 1988 7 6 29,208
Parkway 1989 6 5 19,488
Sugar House 1996 10 9 20,344
Stansbury 1999 7 6 24,145
Redstone 2001 7 5 26,309
South Jordan 2003 3 2 11,359
Centerville 2007 4 4 8,044

Care by DesignTM
• Appropriate Access – 2003
• Balance supply and demand of visits
• Standardized schedules
• Care Team – 2004
• Expanded MA role
• Providers and MAs working in teams
• EMR tools
• Planned Care – 2006
• Pre-visit planning
• Registries
• Labs prior to visit

Retrospective Analysis:
Qualitative Aims
AHRQ Grant # 1R18 HS019136-01 Transforming Primary Care
Aim Method
Document and measure the
transformation
• Archival Search
Determine impact on the
experiences and satisfaction
of providers, staff and
patients
• Provider and Staff Surveys
• Provider and Staff Interviews
• Patient Satisfaction Survey
• Patient Focus Groups
Explore organizational &
contextual factors
• Clinic Environmental Audit
• In-Clinic Observations
Assess in depth how the
transformation was
implemented
• Archival Search
• Leadership Interviews
• Provider and Staff Interviews

Care by DesignTM
• Appropriate Access – 2003
• Balance supply and demand of visits
• Standardized schedules
• Care Team – 2004
• Expanded MA role
• Providers and MAs working in teams
• EMR tools
• Planned Care – 2006
• Pre-visit planning
• Registries
• Labs prior to visit

Qualitative Data: Care Teams
8
Component Type of Information Gathered
Archival search • when/how the care team was rolled out
Clinic
Environmental Audit
• size of clinic, team composition, patient volume,
presence of specialists
In-clinic observations • feeling in the clinic, background info
Employee Interviews • personal experience with implementing care
team + experimenting with local adaptations:
how + why
Leadership interviews • personal experience with leading the care team
roll out + managing the evolution: what + why
Provider & Staff
Survey
• trends in team development, employee burn out,
organizational culture
Patient Sat. Surveys • patients’ satisfaction with visits
Patient Focus Groups • changes noticed and patient perspective

Care Team Structure
& MA Role CBD Care
Team
Model
Variations
Traditional
Model
Team
Members:
• Providers
• MAs

“Care
Team”
• 5 MAs: 2 Providers
• Working together
• Doing it all!
MA
specialists
• (V1): 1 MA phlebotomist does all
draws
• Others are 5 MAs :2 Providers
• (V2): 1 MA rooms patients + 1 MA
scribes in the room : 1 Provider
Clinic-
wide team
• All of the MAs are in one pool
• Room patients in a rotation
• Outside visit work done in between
Hybrid
Traditional
Model
• 1 MA : 1 Provider
• Variation – 2 MAs : 1 Provider
Team
Members:
• Providers
• MAs
Care Team Structure
& MA Role
• (V1): 5 MAs : 2 Providers for patient visits, but
• 2 MAs: 1 Provider for outside visit work
• (V2): 5 MAs : 2 Providers, but
• 1 “primary” MA : 1 Provider

Example of Insights from
Quantitative Research
Clinic Culture
An illustration of possible
explanations for the observed
differences in implementation
of Care Teams

Organizational Culture
Assessment Instrument: “Competing Values”
Quinn, Rohrbaugh: http://www.ocai-online.com/

Greenwood

Parkway

Redstone

Senior Leadership

Organizational Culture In Community Clinics

Aim Method
Document and measure the
transformation and impact on
the quality of patient care
delivery
• Clinical Data
• CBD Implementation
Determine impact of the
transformation on cost to the
clinics
• Operational Data
Determine impact of
transformation on overall costs
of healthcare services,
including direct costs to
patients
• Centers for Medicare &
Medicaid Services Data
• Utah All Payer Claims
Database
Retrospective Analysis:
Quantitative Aims
AHRQ Grant # 1R18 HS019136-01 Transforming Primary Care

Quantitative Data
Component Type of Information Gathered
CBD Implementation • Use of EMR tools
• Appointment availability
• Continuity with PCP
• Use of pre-visit planning tools and processes
• Flow and processes of Care Team
• Efficiency of visit/wait times
Impact on Operations • Provider productivity
• Financial performance
• Patient population characterization
Clinical Outcomes • Quality performance (chronic & preventive)
• Patient, Provider, Staff satisfaction
Cost of Care • Utilization and cost of care
• CMS
• Utah Population Data Base (UPDB)
• Utah All Payer Claims Database (APCD)
Gray = data analysis pending

10%
20%
30%
40%
50%
60%
70%
80%
2003 2004 2006 2008 2009
Quality Measures
Percent of Patients Receiving Recommended Care
CAD* Preventive Care* Diabetes* Heart Failure*
Note: Sample size=14 providers who had all measures in FY 2003, 2004, 2006, 2008, and 2009; *p≤0.05

20%
30%
40%
50%
60%
70%
80%
90%
100%
2003 2004 2006 2008 2009
Patient Satisfaction
Percent of Patients Reporting "Very Satisfied"
Recommend provider* Explanation of what was done*
Visit overall* Time spent with physician*
Length of time waiting at office*
Note: Sample size=16 providers who had all measures in FY 2003, 2004, 2006, 2008, and 2009; *p≤0.05

Overview of Quantitative Design:
Link data from multiple sources to assess
impact of transformation to CBD
Cost &
Utilization
CBD
Implementation
Clinical
Data
Operations
Data

Level of CBD Implementation:
2008
1.00
1.20
1.40
1.60
1.80
2.00
2.20
All elements

Examples of Correlation between
CBD Implementation and Patient Satisfaction
Patient Satisfaction
CBD Implementation Measure
2008
Same Day Appointments Efficient Visit
Length of time waiting at the office 0.61** 0.33*
Time spent with the physician/health
care professional you saw
0.20 -0.21
Explanation of what was done for you 0.14 -0.13
The visit overall 0.50** 0.20
Would you recommend the
physician/health care professional to
your friends and family?
0.34* -0.17
N=16 providers *p≤0.1, **p≤0.05

Correlation between CBD
Implementation and Quality Measures
CBD Implementation
Measure 2008
Quality Measures
Diabetes
Coronary Artery
Disease
Preventive Care
Seen by PCP last visit 0.60* 0.61* 0.57*
Use of X-files by MA 0.33* 0.18 0.18
Best Practice Alerts 0.34* 0.26 0.29
After-Visit summary 0.23 0.18 0.11
Labs done prior to
visit
0.54** 0.47* 0.36*
N=14 providers with data across five years *p≤0.1, **p≤0.05

Impact of practice redesign
• Quality improves
• Continuity matters
• Pre-visit planning and EMR reminders help
• Patients notice
• Access improves patient satisfaction
• Level of implementation varies across
clinics
• Clinic culture impacts implementation
• Culture is a critical factor in translational
research

Future analysis:
impact of redesign on…
• Internal cost and
productivity of clinics
• Overall utilization
• Total cost of care

Future: Internal Performance Analyses
Cost &
Utilization
CBD
Implementa-
tion
Clinical
Data
Operations Data
• Provider productivity
• Financial performance
• Patient population
characterization

Future: Cost and Utilization
Cost & Utilization
CMS, APCD
CBD
Implementa-
tion
Clinical
Data
Operations
Data
CMS Data
• All Medicare Claims
at individual level for
Utah (2007+)
• For the following:
• Outpatient
• Inpatient
• Home Health
• Nursing Home
• Prescription
Drug (Part D)
•Linked to State Vital
Statistics and facility
data (Utah Population
Database )
All Payer Claims
Database (APCD)
• Data elements:
• Charges
• Reimbursements
• Utilization
• For the following:
• Outpatient,
Inpatient,
Rehabilitation
• Prescription Rx
• Linked to State
Vital Statistics and
facility data

Challenges in Assembling
Cost, Utilization and Demographic Data
• Gaining access
• Navigating layers of documentation,
requests, approvals (CMS)
• Obtaining IRB and other database
approvals
• Building APCD platform as 1st user
• Translating utility into usable research database
• Creating files linkable at individual level
• Linking data – hospital, ED, vital statistics

Challenges of Retrospective
Mixed Methods Research
• Timing of all the components
• Recall isn’t perfect – current events color
memory
• Data used for operations differ from data
required for research
• IRB & HIPAA rules for linking PHI to
operations and external data

Benefits of
Mixed Method Research
• Multiple components inform each other
throughout data collection
• Participant selection
• Instrument development
• Sequencing
• Multiple components inform each other
throughout data analysis
• Convergent/consensual validation
• Multiple components facilitate integration of
different perspectives