The document discusses ensuring quality of health care data from a Canadian perspective. It provides an overview of the Canadian Institute for Health Information (CIHI), which collects health data from various partners across Canada. CIHI faces challenges as a secondary data collector, dealing with varying standards and incomplete data reporting. The document outlines CIHI's strategies to ensure data quality, including its data quality framework, quality reports and studies, and techniques for communicating data quality to different audiences.

1. Ensuring Quality
of Health Care Data:
A Canadian Perspective
Data Quality Asia Pacific Congress 2011
Heather Richards
Consultant
Canadian Institute for Health Information (CIHI)
Tel:+1 250 220 2206
Email: hrichards@cihi.ca
1

2. Agenda
> The Canadian Institute for Health Information
> Data Quality Challenges in Canada
> Strategies to Ensure Data Quality:
– CIHI’s Data Quality Framework
– Data Quality Reporting Tools and Studies
– Techniques for Communicating Data
Quality
2

3. The Canadian Institute for
Health Information
3

4. Canadian Institute for Health Information
> National, independent, not-for-profit agency,
established in 1994
> One of Canada’s leading sources of high-quality,
reliable and timely health information
> 27 health databases
and registries
> 7 offices
4

5. CIHI’s Mandate
> Coordinate, develop, maintain and disseminate
health information on Canada’s health system
and the health of Canadians
5

6. CIHI's Mandate Con't
> Provide accurate and
timely information
required for:
– Sound health policy
– Effective management
of the health system
– Public awareness about
factors affecting
good health
6

7. Data Quality Challenges
in Canada
7

8. Data Challenge: Variety of Partners
> Accommodating different coding standards at
provincial/territorial level versus national level;
> Recognizing different uses of the data and different
focus on data quality;
> Adjusting for differing data collection methods
8

9. CIHI Partners
Health Regional
facilities health authorities
Statistics Health
Canada Canada
Ministries CIHI Professional
of health associations
Non-governmental Private sector
organizations organizations
Researchers
9

10. Data Challenge: Secondary Data Collector
> CIHI does not collect data directly
> Our data comes from:
– provincial governments;
– hospitals; and
– professional associations
… this means that
we cannot affect first hand
how that data is captured and collected.
10

11. Data Challenge: Secondary Data Collector
> CIHI relies on data providers (some are voluntary
data providers) to report accurate information
> Poor quality data often result from difficulties in
collection standards, coding standards and
chart documentation – and lack of training
11

12. Data Challenges: Other
> Variety of databases and usability
> Data flow and timeliness
> Coding and comparability
> Hospital practices and data completeness
12

13. End-stage renal failure
www.vancouversun.com 13

14. Question: Are Risk Factors Completely
Captured at all Facilities?
Prevalence of Pulmonary Edema
70%
60%
50%
40%
Inter-Quartile Range: 13-27%
30%
20%
10%
0%
14

15. Questionnaire Reveals a Correlation of
Data Completeness to Hospital Practices
Prevalence of Pulmonary Edema
70%
60%
50%
40%
IQR: 8-21% IQR: 16-29%
30%
20%
10%
0%
Reviews select Reviews all documentation
documentation
15

16. Chart Review Confirms Under-Reporting
Prevalence (%)
Data Captured
Data Captured
by CIHI coder
by Dialysis
during Chart
Clinic Staff
Review
Pulmonary edema 22 27
Sensitivity=62%
Specificity=93% Epidemiologists and clinical
researchers prefer seeing
PPV=77% these statistics…
NPV=87%
16

17. Strategies to Ensure Data Quality
• CIHI’s Data Quality Framework
• Data quality reports and studies
• Techniques for communicating DQ
17

18. Strategies to Ensure Data Quality
CIHI’s Data Quality Framework
18

19. CIHI’s Data Quality Framework
> Objective approach to
assessing data quality
and producing standard
documentation
> Three parts
1. Work Cycle
2. Assessment Tool
3. Documentation
19

20. 1. Data Quality Work Cycle
Plan
Assess Implement
20

21. 2. Data Quality Assessment Tool
> Provides a consistent
approach for defining
data quality
5
> Five dimensions Dimensions
– Accuracy
– Comparability 19
Characteristics
– Timeliness
– Usability
61
– Relevance Criteria
21

22. 2. Data Quality Assessment Tool
Accuracy
Comparability
Coverage
Timeliness Capture and collection
Usability Unit non-response
Item (partial) non-response
Relevance Measurement error
Edit and imputation
Processing and estimation
Population of reference explicitly stated
Coverage issues are documented
Frame validated
Under or over-coverage rate
22

23. Assessment Tool: Educational Component
23

24. 3. Data Quality Documentation
> Details the data quality
documentation required
for each data holding
24

25. Metadata Documentation
Retain
knowledge
about the
management
of a
database
with the
database.
25

26. Strategies to Ensure Data Quality
Data Quality Reporting Tools
and Studies
26

27. Deputy Minister Data Quality Reports
> Bird’s eye view
> Broad DQ scope:
assess accuracy,
timeliness, comparability
and usability
> Specific audience:
Deputy Ministers of
Health
27

28. Features of the Deputy Minister
Data Quality Reports
> Each indicator is important to the success
of a database and has a defined action to
improve performance
– Snapshot of results across all jurisdictions
– Trending over time
> 11 databases
– 8 from CIHI
– 3 from Statistics Canada
28

29. Components of the Deputy Minister
Data Quality Reports
P/T indicator
tables
Trending Database-
results specific reports
Technical
Flags table
documents
Each DM package
29

30. Trending: Discharge Abstract Database
Indicator 1: Total Outstanding Hard Error
Rate, per 1,000 Abstracts
2.5
2.0
2003-04
1.5
2007-08
2008-09
1.0
2009-10
0.5 2004-05 2005-06
2006-07
0.0
Optimal Value = 0
30

31. Response to Reports
Positive:
> Highlights to DM the value of a
database; increases coverage of
data holdings
> Reveals systemic problems
causing DQ issues; helps Deputy
Ministers prioritize and reallocate
resources
> Congratulates on past DQ
improvements; facilitates creation
of DQ improvement action plans
31

32. Reabstraction Studies
> Detailed review
> Narrow DQ scope: assess
coding consistency,
correctness, completeness
> Wide audience
32

33. Study Methods
> A chart review to
recapture the data and
compare
Reabstractor
assigns
Application reasons for
compares differences
data
Application
reveals original
data
Reabstractor
recodes
from chart
33

34. Overview
Determine
Share study
results method
Study Objectives
Process Develop
and data
analyze collection
data tool
Train
coders,
collect data
34

35. Reabstraction Study Example
DAD: Discharge Abstract Database
> Data on acute-care hospital activity
> Data supports:
– funding and system planning decisions at
government level
– management decisions at the facility level
– clinical research at the academic level
35

36. Strategies to Ensure Data Quality
Techniques for Communicating
Data Quality
36

37. Communicating Data Quality Using
Different Lenses
Statistics for OECD
international Isolating determinants
comparisons of good health
Health Assessing
quality of care
indicators
Clinical
Categorizing research
hospitalizations purposes
for hospital such as
management purposes survival analysis
37

38. Health > Assess population health and
health system performance
Indicators
> Will look at one indicator:
ACSC hospitalizations
38

39. Health Indicator: ACSC Hospitalizations
Age-Standardized Rate of ACSC
Hospitalizations per 100,000 Population
600
500 459
400 2001-02
2002-03 2003-04 326
2004-05 2005-06
300 2006-07
2007-08
200
100
0
39

40. 2007-08 DAD Study: ACSC Hospitalizations
> Question: Is the decrease in ACSC
hospitalizations real or is it due to changes in
coding quality?
> Answer: The observed decrease is real
– National rates are indeed decreasing
– Reabstraction studies found that certain patient
populations had lower quality data
40

41. 2007-08 DAD Study: ACSC Hospitalizations
Sensitivity
Grand mal status, epileptic convulsions 81%
Chronic obstructive pulmonary diseases 91%
Asthma 90%
Diabetes 95%
Heart failure and pulmonary edema 84%
Hypertension 100%
Angina 94%
Any ACSC hospitalization 90%
41

42. Data Quality Challenges that Lie Ahead
> The health sector is a changing landscape
– Electronic health record
– Health care funding
– New technologies
– New modes of delivering
health care
> New data will bring new quality challenges
42

43. “Taking health information further”
43