PhD defense presentation

1. PhD Defense, October 9, 2009
ENHANCING CONTINUITY OF CARE USING
EMERGENCY MEDICAL CARD AND
CONTINUITY OF CARE REPORT
Christopher H O Olola MSc
Department of Biomedical Informatics
University of Utah
Committee:
R Scott Evans PhD (Chair)
Mollie Poynton APRN,PhD
Scott Narus PhD
Jonathan Nebeker MD
Joseph Hales PhD

2. PhD Defense 2October 9, 2009
PRESENTATION OUTLINE
Introduction
Background
Project implementation
Project evaluation
Conclusions

3. PhD Defense 3October 9, 2009
PRESENTATION OUTLINE
Introduction
Background
Project implementation
Project evaluation
Conclusions

4. PhD Defense 4October 9, 2009
CURRENT HEALTHCARE SYSTEMS
Healthcare systems are complex and fragmented1,2
Partly due to slow EMR implementation and adoption3
USA: 17%-20% Outpatient clinics have comprehensive EMRs
Results in ineffective patient care
Patients see multiple healthcare providers (HCPs)
Mostly because of changes in insurance plans, care
quality, care costs, loss of trust etc
Potentially results in discontinuity of care
≈ 44,000 – 98,000 Americans die annually
These deaths occur because of (mostly avoidable) medical
errors and ADEs as a result of poor quality information4,5
1Geissbuhler et al. 2004, 2Bates et al. 2003, 3HIMSS 2006, 4Kohn et al. 2000, 5Moore et al. 2003

5. PhD Defense 5October 9, 2009
CONTINUITY OF CARE (COC)
What is Continuity of Care?
A process by which the patient and the physician
are cooperatively involved in ongoing health care
management toward the goal of high quality, cost-
effective medical care.
American Academy of Family Physicians – AAFP (2003)

6. PhD Defense 6October 9, 2009
TYPES OF CONTINUITY OF CARE*
Interpersonal or provider continuity
Where a patient has a single HCP with whom s/he has
developed trust, respect and understanding
Longitudinal or site continuity
Where a patient has groups of preferred HCPs that s/he sees
for medical care i.e., Team-based or “medical home”
Informational or referral continuity
Where there exists organized and shareable patient core
information among HCPs
This is the focus of my project…
*Freeman et al. 2003, Roos et al. 1980, Saultz et al. 2003 HCP: Healthcare Provider

7. PhD Defense 7October 9, 2009
TECHNIQUES FOR PROMOTION OF COC*
Use of a single primary HCP
Directly observed treatment (DOT)
Bar codes
Radio frequency identification (RFID)
Regional Health Information Organizations (RHIOs)
Pocket-sized smart card
Emergency medical cards (EMCs)
Continuity of care record (CCR)
This is the focus of my project…
*Fry et al. 2005, Haung et al. 2005, Orlova et al. 2005, Auber 2001

8. PhD Defense 8October 9, 2009
PRESENTATION OUTLINE
Introduction
Background
Project implementation
Project evaluation
Conclusions

9. PhD Defense 9October 9, 2009
INSTITUTIONAL REVIEW BOARDS (IRBS)
All the studies in this project were
approved by the University of Utah IRB
and the Intermountain Healthcare IRB

10. PhD Defense 10October 9, 2009
PROBLEMS/SIGNIFICANCE OF PROJECT
Patients see multiple HCPs outside of Intermountain
Healthcare network
Dearth of immediately available patient healthcare
information at the point of care
Lack of standardized patient care information to
share across health systems
Poor quality (accuracy, completeness) patient data
Healthcare inefficiencies
In most instances CoC is rarely considered*
*Mills et al. 2006, Post et al. 2005 HCPs: Healthcare Providers CoC: Continuity of Care

11. PhD Defense 11October 9, 2009
PROJECT OBJECTIVES
1. Develop an automated application (“CCR application”)
compliant with the CCR standard (E2365-05)*
2. Use simulation to assess the use and usefulness of the
EMC and CoC report in enhancing CoC
3. Evaluate the impact of patient-entered data on the
quality (accuracy and completeness) of HCP-entered
data in the EMR
4. Assess patient satisfaction with usefulness of the EMC
and CoC report in enhancing CoC
*ASTM 2006 CoC: Continuity of Care CCR: Continuity of Care Record

12. PhD Defense 12October 9, 2009
CONTINUITY OF CARE RECORD (CCR) STANDARD
XML: eXtensible Mark-up Language DPH: Department of Public HealthASTM 2006
HeaderBodyFooter
Developers: ASTM & other organizations
XML-based standard
Outgrowth of Massachusetts DPH
patient care referral form (PCRF)
Core dataset of the most relevant and
timely facts on patient’s health care
Originally designed to be prepared by a
practitioner at conclusion of encounter
To enable next practitioner to readily
access pertinent patient information
CCR may be prepared, displayed and
transmitted on paper or electronically
CCR is not EHR, Progress note, Discharge
summary

13. PhD Defense 13October 9, 2009
PRESENTATION OUTLINE
Introduction
Background
Project implementation
Project evaluation
Conclusions

14. PhD Defense 14October 9, 2009
IMPLEMENTATION OF EMERGENCY MEDICAL CARD
AND CONTINUITY OF CARE REPORT FOR
CONTINUITY OF CARE
Olola CHO, Rowan B, Narus S, Smith M, Hastings T, Poynton M,
Nebeker J, Hales J, Evans RS
Methods of Information in Medicine
(Accepted for publication: May 2009)

15. PhD Defense 15October 9, 2009
STUDY OBJECTIVES
1. To describe the procedures used to design,
develop, and implement the CCR application,
EMC and CoC report using the CCR standard
2. To outline the evaluation studies planned and
the major lessons learned
EMC: Emergency Medical Card CoC: Continuity of Care CCR: Continuity of Care Record

16. PhD Defense 16October 9, 2009
METHODS
Setting and Users
Develop and integrate the CCR application in the My
Health patient portal at Intermountain Healthcare
The application is used by approximately 30,000
patients enrolled at 25 outpatient clinics that offer My
Health web services
The sites have over 120 Internal Medicine, Family
Practice, OB/Gyn, diabetes and Endocrinology
clinicians
The project took 2.5 years (December 2006 – October
2008) to implement, evaluation studies 9 months

17. PhD Defense 17October 9, 2009
METHODS
The Data Management Architecture
Phase 1 (CCR application implementation)
1a/b: Data entry and display
2: EMC data transformed into XML & stored
3: CCR XML data are extracted from CDR
4: pdf of EMC/CoC report generated
5a/b: EMCs /CoC reports printed or saved
6: EMCs/CoC reports used with primary HCPs
7: EMR updated with patient-entered data
Phase 2 (for future implementation)
step 8: HCPs extract patient-entered data
step 9: HCPs manages data discrepancies
step 10: HCPs verifies data and updates EMR

18. PhD Defense 18October 9, 2009
METHODS
The Main Data Management Screen

19. PhD Defense 19October 9, 2009
The Emergency Medical Card
8-faced foldable card
Designed using ISO 7810
ID-1 standard (ATM card
size)
Contains current patient-
entered data & EMR data
(not differentiated)
Used mainly during
medical emergency

20. PhD Defense 20October 9, 2009
The Continuity of Care Report
Header
Contains patient-entered
data and HCP-entered data
 All active problems,
allergies, medications etc
are included
 3 months back (plus
inactive) problems, meds,
labs etc included
Used mainly during non-
emergency clinic visits or
taken to patient primary
provider to update missing or
erroneous data in EMR

21. PhD Defense 21October 9, 2009
RESULTS AND DISCUSSIONS
An application complaint with the CCR standard
requirements was designed, developed and integrated
with Intermountain’s electronic PHR, My Health
Patients use online credentials to access their PHRs to
View, add or modify their PHRs
Create and print paper-based EMC & CoC report using both
patient-entered & HCP-entered EMR data
Monitor records, identify possible errors and (using the CoC
report) communicate to HCPs for review and EMR update
Two evaluation studies were designed to assess &
report on the application using simulation, reviews and
comparisons of EMC/CoC report and EMR data, and
patient-satisfaction surveys

22. PhD Defense 22October 9, 2009
LESSONS LEARNED & STUDY LIMITATIONS
EMR data update done only by primary HCPs at
Intermountain is limiting – delays EMR updates &
information availability at point of care
Patient proxy are needed e.g., if patient is incapacitated
Currently, HCPs have no direct access to patient-entered
data in the CCR application database
Keeping the EMC data current even if by using the
unverified patient-entered data is vital
PHC: Personal Health Console

23. PhD Defense 23October 9, 2009
CONCLUSIONS
Demonstrated that it is possible to use CCR standard
to implement an application that enables patients,
not only to view their PHRs but to add or modify
records, & to create and print EMCs and CoC reports
EMCs/CoC reports can be created using the HCP-
verified EMR data & not by using patient-entered data
only as is currently prevalent in healthcare systems
Functionalities that enable patients to monitor their
records, identify possible errors & communicate to
HCPs for prompt EMR updates, are crucial

24. PhD Defense 24October 9, 2009
PRESENTATION OUTLINE
Introduction
Background
Project implementation
Project evaluation
Conclusions

25. PhD Defense 25October 9, 2009
USE OF SIMULATION TO EVALUATE THE USE AND
USEFULNESS OF THE EMERGENCY MEDICAL CARD
AND CONTINUITY OF CARE REPORT IN ENHANCING
CONTINUITY OF CARE
Olola CHO, Narus S, Nebeker J, Poynton M, Hales J, Rowan B,
LeSieur H, Zumbrennen H, Edwards AA, Crawford R, Amundsen S,
Kabir Y, Atkin J, Newberry C, Young J, Hanifi T, Risenmay B, SorensenT,
Evans RS
Methods of Information in Medicine
(Submitted: September 2009)

26. PhD Defense 26October 9, 2009
STUDY OBJECTIVES
1. To use simulation to evaluate the use of the EMC and
the CoC report in enhancing CoC
2. To assess the usefulness of the EMC and the CoC
report in enhancing CoC
EMC: Emergency Medical Card CoC: Continuity of Care

27. PhD Defense 27October 9, 2009
METHODS
Setting and participants (“Reviewers”)
3 Medical Doctors (MDs) & 2 Physician Assistants (PAs)
from outpatient clinics at Intermountain Healthcare
Clinic Managers at the Intermountain clinics made
contacts
7 Fourth-year medical students from the University of
Utah School of Medicine
PI made contacts

28. PhD Defense 28October 9, 2009
METHODS
Cases of patients who
entered new data using
CCR application
De-identified
study cases
(n=3)
EMC CoC reportEMR
5 Complete an online
survey (with Likert
Scale responses)
4Review
2 Create
1Random
selection
3Review
• Legibility, easy to use & understand
• Encounter time, overall HCP knowledge
• Medical decision making

29. PhD Defense 29October 9, 2009
RESULTS AND DISCUSSIONS
Measure n
Usefulness (agree/useful responses) in enhancing CoC
EMC: n(%) RD(95%CI) p CoC report: n(%) RD(95%CI) p
Sex
Female 16 14(87.5) 11(68.8)
Male 15 15(100.0) 0.13(-0.04,0.29) 0.484 12(80.0) 0.11(-0.19,0.42) 0.685
Designation
MD/PA 13 11(84.6) 8(61.5)
Medical students 18 18(100.0) 0.15(-0.04,0.35 0.168 15(83.3) 0.22(-0.10,0.53) 0.228
Specialty
Family practice 8 6(75.0) 5(62.5)
Internal medicine 3 3(100.0) 0.118 1(33.3) 0.086
Medical students 20 20(100.0) 17(85.0)
Age of HCPs (years)
18-30 21 21(100.0) 18(85.7)
31-40 4 2(50.0) 0.026 3(75.0) 0.019
41-50 3 3(100.0) 0(0.0)
51-60 3 3(100.0) 2(66.7)
Job experience (years)
(mean ± SD)
31 29(5.31±7.68)
2(10.0±0.00)#
0.402 23(5.09±7.25)
8(7.13±8.54)#
0.518
RD(95%CI): Risk Difference (95% Confidence Interval) ) #Tool rated as not useful
Gender, job designation, specialty and experience had no significant influence in
the way the reviewers evaluated the usefulness of the EMC and CoC report
Ratings significantly varied by reviewer’s age

30. PhD Defense 30October 9, 2009
RESULTS AND DISCUSSIONS
The reviewers provided useful comments on how the
EMC/CoC report could be improved to avail adequate
and appropriate information for effective medical
decision making at the point of care “too many headers - condense and just list address, phone #s, no need for country”
 “information is a bit confusing”
 “health concerns and medications did not match - I guess this is putting the provider at fault for
not adding Gout, HTN and depression to their problem list”
 “Not enough information about chief complaint to be able to answer this question”
 “The only important/likely pertinent health concern for this patient was hx of sepsis, rest had
nothing to do with pt's plan/med probs, etc.”
 “Get rid of unnecessary vital signs; pulse oximetry should be just pulse, no need for resp rate,
temp, PEFR (unless has asthma); abbreviate ht and wt; get rid of pending appts, resolved
allergies, past health concerns, and condense past appts to one line with reason/date”
 “Condense info on CCR to 1-2 pages per patient so easy to use and worthwhile to sort through”
Reviewers preferred condensed/summarized and abbreviated information

31. PhD Defense 31October 9, 2009
RESULTS AND DISCUSSIONS
Inter-rater agreement
Measures
Interpretation/Agreement (n)*
EMC CoC Report
Using the document
Document legibility 0.49 0.58
Document understandability 0.71 0.41
Shortening of encounter time with the patient 0.47 0.80
Increase of healthcare provider’s overall knowledge 0.30 0.54
Influence of the knowledge gained from the data, on the decision to change
Patient diagnosis 0.34 0.80
Patient prescription drugs or therapy 0.28 0.40
Recommendation to discharge the patient home 0.54 0.54
Recommendation to admit the patient 0.33 0.61
Recommendation to repeat the laboratory tests 0.38 0.41
Recommendation to order new laboratory tests 0.25 0.54
Recommendation to refer the patient 0.63 0.69
Recommendation to transfer the patient 0.89 0.75
Overall mean κ 0.47 0.59
Overall moderate reviewer agreement (50%-60%) on rating of EMC & CoC report
Higher agreements observed in individual measures, especially for CoC report
*Kappa statistics (κ)
< 0 Poor or None 0.00—0.20 Slight 0.21—0.40 Fair 0.41—0.60 Moderate
0.61—0.80 Substantial 0.81—1.00 Almost perfect.

32. PhD Defense 32October 9, 2009
Measures
Usefulness: n=31
EMC
(%)
COC Report
(%)
Document legibility & understandability 88.7 90.3
Shorten encounter time & Increase HCP’s overall knowledge 100 100
Patient diagnosis 67.7 67.7
Patient prescription drugs or therapy 93.6 93.6
Recommendation to repeat the laboratory tests 67.9 75.0
Recommendation to order new laboratory tests 87.1 90.0
Recommendation for patient disposition* 63.4 75.0
Overall 81.2 84.5
RESULTS AND DISCUSSIONS
*Discharge home, admission, referral or transfer to another HCP
Overall, EMC and CoC were found to be highly useful (81.2% vs. 84.5%, resp.)
100% usefulness in shortening encounter time & in increasing HCPs’ knowledge
Both were rated highly for legibility and ease of understanding (88.7% vs. 90.3%)

33. PhD Defense 33October 9, 2009
STUDY LIMITATIONS
Few MDs and PAs participated (no NPs)
Decliners were busy, in other projects or not interested
Non-Intermountain HCPs were excluded
Policies prohibited analysis of data outside of Intermountain
network (non-covered entities)
Marketing/recruitment policy restrictions
Intermountain policies permitted only clinical managers at
the clinics offering My Health services to recruit study
reviewers –direct contact PI may be could have improved
recruitment rates

34. PhD Defense 34October 9, 2009
CONCLUSIONS
The EMC and CoC report are useful vehicles for
transporting patient healthcare information across the
healthcare continuum and they can substantially
enhance CoC. This was specifically demonstrated in
Shortening patient-HCP encounter time
Increasing the HCP overall knowledge
Decision on prescriptions and on ordering new or repeating
laboratory tests
The reviewers’ perception of the usefulness of the EMC
and CoC report in enhancing CoC was associated with
age – further (larger) validation studies are needed

35. PhD Defense 35October 9, 2009
ASSESSING PATIENT SATISFACTION WITH THE
CONTINUITY OF CARE AND EFFECT OF PATIENT-
ENTERED DATA ON THE QUALITY OF HEALTHCARE
PROVIDER-MAINTAINED EMR DATA
Olola CHO, Poynton M, Hales J, Narus S, Nebeker J, Rowan B, Smith
M, Evans RS
The International Journal for Quality in Health Care
(Prepared for submission)

36. PhD Defense 36October 9, 2009
STUDY OBJECTIVES
To evaluate patient satisfaction with the usefulness of
the emergency medical card (EMC) and CoC report in
enhancing CoC
Compared patient-entered data in the CoC report with
the HCP-entered data in the EMR data to assess EMR
data quality (i.e., accuracy and completeness)

37. PhD Defense 37October 9, 2009
METHODS
Patient satisfaction survey
Promotional emails, meetings, “Teaser”, fliers, posters
Excluded – never used
EMC/CoC report
(n=32)
Complete online survey
(with Likert Scale)
(n=101; 76%)
• Legibility, easy to use & understand
• Encounter time, overall patient’s knowledge
• Correct errors & complete missing data in the EMR
• Overall quality of care
Users of the CCR
application to create
EMC/CoC report
(n=133)

38. PhD Defense 38October 9, 2009
METHODS
Evaluation of accuracy of HCP-entered data in EMR
Instances of patient-
entered data values
(n=1,994)
Study sample
(n=1,505, 75.5%)
Excluded – had no
corresponding data
fields in the EMR
(n=489)
Non-repeating records
(e.g., address)
Repeating records
(e.g., labs, vitals)
EMR data
Summary of instances
of accurate data values

39. PhD Defense 39October 9, 2009
METHODS
Evaluation of completeness of HCP-entered data in EMR
Instances of all data
values (8 months before
use of CCR application)
Summary of instances
of complete data values
Instances of all data
values (8 months after
use of CCR application)
Instances of data
values with non-blank/
missing values
Instances of data
values with non-blank/
missing values
Compared only data fields
that existed before and after
use of CCR application

40. PhD Defense 40October 9, 2009
RESULTS AND DISCUSSIONS
Cumulative CCR application’s page views & actual use
Promotional techniques significantly increased number of the application’s page views
(n=22,024) and actual use (n=133)
Strong correlation between page views and actual use (γ = 0.994, p=0.0005)
Email messages to HCPs was best, then “Teaser” in My Health & lastly Fliers to HCPs

41. PhD Defense 41October 9, 2009
RESULTS AND DISCUSSIONS
Documents’ usefulness rating was significantly associated with patient’s age
Majority created EMC for use in emergencies (56%) or for personal use (26%)
Only about 15% of patients created CoC report for EMR updates
Measurement
Patients (N=101)
n(%)
Usefulness: n(%)
EMC p† CoC report p†
Sex Female 76(75.3) 41(54.0)
1.000
48(63.2)
0.232
Male 25(24.7) 13(52.0) 12(48.0)
Age group (years)
20-30 12(11.9) 7(58.3) 9(75.0)
31-40 14(13.9) 10(71.4) 11(78.6)
41-50 22(21.8) 11(50.0) 0.019 10(45.5) 0.009
51-60 35(34.7) 12(34.3) 15(42.9)
>60 18(17.8) 14(77.8) 15(83.3)
Use of EMC and CoC report*
To update records 16(9.4)
To correct data errors 9(5.3)
For personal use 44(25.9)
Stored for emergency use 96(56.4)
Reviewed and discarded 2(1.2)
Others 3(1.8)
Profile of responses
*Data was collected in an Exit survey after patients created the documents

42. PhD Defense 42October 9, 2009
RESULTS AND DISCUSSIONS
Patients’ rating of the usefulness of EMC & CoC report
EMC CoC report
Overall
pMeasure n
usefulness
n(%)
Score
Mean±SD n
usefulness
n(%)
Score
Mean±SD
Data accuracy 99 71(71.7) 3.8±1.1 100 74(74.0) 3.8±1.0 0.882
Error correction 90 58(64.4) 3.8±1.0 92 62(67.4) 3.8±1.0 1.000
Completion of missing data 91 68(69.7) 3.9±1.0 94 70(74.5) 3.7±1.0 0.469
Document user-friendly 101 81(80.2) 4.0±0.9 101 84(83.2) 3.8±0.9 0.829
Confidentiality and security 101 84(83.2) 4.1±0.7 101 84(83.2) 4.1±0.7 0.829
Increased knowledge of condition 101 71(70.5) 3.8±1.1 100 69(67.0) 3.8±1.1 0.372
Enhanced quality of care 96 55(57.3) 3.7±1.0 95 59(62.1) 3.7±1.0 0.567
Increased trust with HCP 96 56(58.3) 3.7±1.0 95 55(57.9) 3.6±1.0 0.223
Improved relationship with HCP 95 53(55.8) 3.6±1.0 95 52(54.8) 3.6±1.0 0.203
Shorten encounter time 94 45(47.9) 3.5±1.0 94 42(44.7) 3.5±1.0 0.074
Lengthen encounter time 94 20(21.3) 2.9±1.0 93 23(24.7) 2.9±1.0 0.849
Overall 1,058 662(62.6) 1,060 674(63.6) 1,336(63.1)
Overall, 63.1% of the patients found EMC and CoC report to be useful
More patients for CoC than EMC (64% vs. 63%) , but no significant difference
Agreement on the usefulness of EMC and CoC report was demonstrated in each
CoC measure , but no significant difference in each measure’s ratings
Patients rated documents highly for usefulness in shortening encounter time

43. PhD Defense 43October 9, 2009
RESULTS AND DISCUSSIONS
Patients-entered data vs. accuracy of EMR data
69% (70/101) of the patients entered new data values (n=1,994; 1,505 compared)
44% of new data was used for EMR update, but no significant difference (p=0.109)
The majority of the EMR updates were for address, biodata, insurance, Primary HCP, labs
Significantly low EMR updates observed in Biodata and primary HCP data
No EMR updates for allergies & problems data
Data category Data attributes
Patient entries (n=1,994) EMR-updates
p
n Median(Q1,Q3)# n(%) Median(Q1,Q3)#
Demographic Address 271 5(1,6) 241(88.9) 5(1,6) 0.085
Biodata¶ 555 9(8,9) 240(43.2) 3(3,4) <0.001
Insurance 115 2(1,2) 73(63.5) 1(1,2) -
Primary HCP 121 2(2,2) 77(63.6) 1(1,2) <0.001
Clinical Allergies 106 5(3,5) 0(0.0) 0(0.0) -
Problems 60 4(4,4) 0(0.0) 0(0.0) -
Vital signs 63 6(3,7.5) 18(28.6) 2(2,4) 0.240
Appointments - - - - -
Laboratory Observations & results 5 2.5(1,4) 4(80.0) 4(4,4) 1.000
Medication Prescription drugs 209 6(5,10) 6(2.9) 6(6,6) 0.790
Overall All data categories 1,505 2(2,6) 659(43.8) 2(1,4) 0.109
Q1/Q3: 25th & 75th Quartiles

44. PhD Defense 44October 9, 2009
RESULTS AND DISCUSSIONS
Patient-entered data vs. completeness of EMR data
Category Data attributes Period n Complete data: n(%) OR(95%CI) p
Demographic
Address Before 3,153,789 2,867,118 (90.9)
After 8,488,873 7,991,872(94.2) 1.61(1.60,1.62) <0.001
Biodata Before 24,462,684 23,883,032(97.6)
After 7,319,910 5,943,165(81.2) 0.11(0.10,0.11) <0.001
Insurance Before 5,167,798 4,385,806(84.9)
After 5,714,498 4,840,141(84.7) 0.99(0.98,0.99) <0.001
Primary HCP Before 773,742 773,742(100.00)
After 843,481 843,481(100.0) - -
Clinical
Allergies Before 1,058,023 1,045,856(98.9)
After 1,121,247 1,106,809(98.7) 0.89(0.87,0.91) <0.001
Problems Before 1,247,300 945,565(75.8)
After 1,328,425 1,003,920(75.6) 0.99(0.98,0.99) <0.001
Vital signs Before 43,728,286 43,728,286(100.0)
After 45,632,968 45,632,968(100.0) - -
Appointments Before 20,922,920 20,608,315(98.5)
After 16,199,284 15,985,094(98.7) 1.14(1.13,1.15) <0.001
Laboratory Observation & results Before 277,840,260 276,182,990(99.4)
After 293,902,286 207,256,687(70.5) 0.01(0.01,0.01) <0.001
Medication Before 12,804,015 11,525,098(90.0)
After 16,284,045 14,624,329(89.8) 0.99(0.98,0.98) <0.001
Inconsistency in associations shown between use of CCR application and data completeness
Significant association between use of CCR application and improved data completeness was
observed only in address and appointments data attributes
Overall, 98.7% complete data before & 76.9% after (OR(95%CI: 0.780(0.779, 0.780), P<0.001)

45. PhD Defense 45October 9, 2009
STUDY LIMITATIONS
Some patient data were not extracted from EMR
These were annotated by patients because the data was in
free text format or not integrated in CDR
Some sub-group analyses on EMR data quality did not
yield results due to sample size issues
E.g., Insurance, allergies and medical problems – but these
may be because they accurately entered in EMR by HCPs
Reason was not collected for data not updated in EMR
Opportunity was missed of asking patients reasons why
some entered data was not used to update EMR information

46. PhD Defense 46October 9, 2009
CONCLUSIONS
Patients demonstrated that the EMC and CoC report
were useful tools in enhancing CoC
Patients also showed that the two documents were
useful to identify missing patient information
especially in the demographic and appointment
categories than other categories
The patient is an important source of quality control
for their record in HCP-maintained EMR

47. PhD Defense 47October 9, 2009
PRESENTATION OUTLINE
Introduction
Background
Project implementation
Project evaluation
Conclusions

48. PhD Defense 48October 9, 2009
RESEARCH CONCLUSIONS
Contribution to the field of Biomedical Informatics
Showed that the CCR standard can be used to ensure that patient
information that is shared among HCPs is standardized
Demonstrated that HCP-verified EMR data can be used to create
EMCs (and CoC reports), in addition to the prevalent use of patient-
entered data
Showed that patients are able to access their PHRs, monitor and
control the quality of their information in the HCP-entered data in
EMR
Showed that timely and accurate patient information can be
availed to foster enhanced efficiency and effectiveness in the EMR
updates and medical decision making by HCPs

49. PhD Defense 49October 9, 2009
RESEARCH CONCLUSIONS
Lessons Learned and Future Direction
Free-text, non-integrated data sources and lack of full
interoperability of EMRS still pose data extraction difficulties
Patient is an important source of quality control for their records in
HCP-entered data in the EMR
The EMC and CoC report should be kept up to date since both HCPs
and patients have shown the usefulness of the two documents in
enhancing CoC – more patients printed EMC
Use of a variety of promotional techniques are useful for improved
(wide scope) marketing and recruitment
Further validation studies are recommended

50. PhD Defense 50October 9, 2009
ACKNOWLEDGMENTS
INSTITUTIONAL REVIEW BOARDS (IRBS)
University of Utah IRB
Intermountain Healthcare IRB
NIH/NLM/MICHIGAN STATE UNIVERSITY
Julia Royall
Terrie Taylor
DBMI
Reed Gardner, Joyce Mitchell, Lynn Ford,
John Hurdle
Matt Samore, Adi, Lisa-Canon Albright
Kathy Stoker, Linda Galbreath, JoAnn
Thompson
Faculty, staff and fellow students
ITS/EDW TEAM
Jim Livingston, Cheri Hunter, Ming Tu and
Vick Deshmukh
NATIONAL ACADEMY OF EMERGENCY DISPATCH
Jeff Clawson, Alan Fletcher, Pam Stewart
INTERMOUNTAIN HEALTHCARE
PHC team:
Belle Rowan (Chair), Matt Smith, Traci
Hastings, Carol Askew, Chris Nuccitelli
Len Bowes, Stan Huff
ISSA, EDW, SelectHealth…
UOU/DBMI COMMITTEE
R Scott Evans (Chair)
Mollie Poynton
Scott Narus
Jonathan Nebeker
Joe Hales
MY FAMILY
Mom, Dad, and my Wife and Children