1. Research Methods in Health
Chapter 15. Selected Research Papers
- Purpose, Framework, Results -
Young Moon Chae, Ph.D.
Graduate School of Public Health
Yonsei University, Korea
ymchae@yuhs.ac
2. Table of Contents
• Online psychological service for health professional --------------------------- 3
• Evaluation of CDSS for Drug Prescription Based on Success Measures --12
• Implementing Health Management Information Systems: Measuring
Success in Korea’s Health Centers ------------------------------------------------- 20
• Risk Factors for Cervical Cancer among Korean Women from the Korean
Cancer Prevention Study -------------------------------------------------------------- 30
• Evaluation of Mobile Phone-based Diet Game for Weight Control ---------- 51
• Management Issues in Health Information Systems ---------------------------- 58
• Competency-based Learning for Distance Education in System Analysis
and Design ---------------------------------------------- ---------------------------------- 62
2
3. Burnout, job stress and job satisfaction: A study
among doctors and nurses of Ulaanbaatar,
Mongolia
3
Ariunsanaa B, Sugarmaa M, Khuderchuluun N, Saranchuluun O,
Chae YM
4. Introduction
• Background
– Mongolia is considered to be one of the countries with a sufficient
supply of health professionals in terms of proportion to the size of
the population, especially in urban areas.
– However, the problem of dense population in urban areas and
internal migration has deteriorated the quality, accessibility, and
load of healthcare services.
– In this situation, it is claimed that the stress and depression driven
by over workload of health professionals has somewhat negativel
y influenced their attitude towards work and job satisfaction.
– Other factors, such as the number of the patients, duration of
medical checkups, procedures, consultations, and other
additional functions, like serving patients with special needs, can
lead to significant job stress.
• Significance
– There was no study dealing with job stress and job satisfaction for
health professionals in Mongolia
4
5. Research Objectives
• General
– To study job stress, burnout and some depressive
symptoms of medical doctors and nurses of
tertiary health care settings of Ulaanbaatar
• Specific Objectives
– To determine the prevalence of job stress and burnout
among medical doctors and nurses,
– To determine the prevalence of job dissatisfaction and
some depressive symptoms of medical doctors and nurses
– To analyze the association between job stress, burnout
and depressive symptoms
5
6. Methods
• Study design
– Study design is cross sectional and pre-experimental as it is
social science research.
• Sampling
– In this study 400 of medical doctors and nurses will be included.
– Stratified sampling by using hospital size as strata
6
7. Research Framework
Burn out
Job stress
Job satisfaction
Age
Sex
Education
Job characteristics
Multiple regression
Chi square or t-test
Correlation or
Chi square test
7
8. Analytic Methods
• All variables are ordinal scale (5 points)
• Simple regression vs correlation (direction)
• Correlation vs Chi-square test
• T-test vs Chi-square test
• T-test vs paired t-test
• Parametric test vs non-parametric test
• Multiple regression vs logistic regression
8
12. ANCOVA
12
Table 9. Result of ANCOVA for job stress components (ERI model) and for burnout dimensions
Male MD (n=31)
Female MD (n=1
28)
Female nurses (n
=199) P-valueª
Total (n=362)
Mean SD Mean SD Mean SD Mean SD
Extrinsic effort † 12.03 3.88 13.49 3.79 11.72 3.69 0.000 12.38 3.82
Reward† 47.75 5.75 45.44 5.86 47.35 5.92 0.006 46.70 5.94
Effort-reward ratio† 0.58 0.29 0.68 0.28 0.57 0.27 0.002 0.61 0.28
Overcommitment † 13.60 3.97 13.64 3.47 14.96 3.42 0.002 14.36 3.54
Personal burnout 39.33 17.99 45.35 16.10 44.92 16.61 0.158 44.61 16.57
Work related burnout 40.36 18.43 46.82 18.28 43.24 17.78 0.059 44.28 18.08
Client related burnout †‡ 25.86 19.23 36.31 19.89 30.33 19.72 0.004 32.14 19.97
Job satisfaction†‡ 74.83 14.36 66.67 12.44 70.21 14.88 0.006 69.29 14.15
† Difference between Female MD and Female Nurses (Multiple Comparisons, Tukey HSD)
‡ Difference between Male MD and Female Nurses (Multiple Comparisons, Tukey HSD)
13. Evaluation of CDSS for Drug
Prescription Based on Success Measures
Jinwoo Park , Young Moon Chae, Young Taek Lee, Koungwon Cho,
Junghee Kim, Byung Hwa Lee
(Published in Journal of Korean Society of Medical Informatics.
2009; 15(3): 293-301)
13
14. Introduction
• Background
– Medication errors are increasing
– The Clinical Decision Support System (CDSS) for drug
prescriptions was developed by integrating the computerized
physician order entry (CPOE) system to support doctors and
pharmacists in making correct decisions on prescribing drugs
in line with the prescription guidelines by the Health Insurance
Review Agency (HIRA)
• Significance
– This is the first attempt to evaluate effectiveness of CDSS for
drug prescription in Korea
– Use of DeLone and McFarlan model for performance evaluation
of CDSS
14
15. Objectives
• The purpose of this paper is to analyze the performance
of the CDSS with respect to system quality, information
quality, and user satisfaction to reduce prescription error
• Specific objectives are:
– To identify the factors influencing CDSS performance
– To analyze association between quality measures and user
satisfaction
15
16. Subjects
• A questionnaire was designed by structuring primary
evaluation areas first and then more detailed sub-areas
in each primary area.
• The survey was conducted by mail or by visit for 6 weeks
from April 20 to May 29, 2009, for the pharmacists from
38 hospitals using the CDSS.
• Hospitals were selected by stratified random sampling
• Research design is cross-sectional design
• A total of 84 questionnaires were returned from 22 users
(response rate of 58%). Of them, 77 were used for the
analysis, excluding 7 that were found unusable for
statistical analysis.
16
17. Model and Statistical Analysis
• Based on the Delone and McLean framework, we evaluated
the success of the CDSS by using two quality measures
(system quality and information quality) as independent
variables; and three performance measures (user satisfaction
and organizational impact) as dependent variables for the
multiple regression analysis (Figure 1).
• Chi-square analysis with cross-tabulation was performed to
analyze the specific associations among two quality measures
and three measures of user satisfaction (information
satisfaction, system satisfaction, and willingness to
recommend to others).
• We also analyzed the associations between two quality
measures and improvements in drug safety as a performance
measure of organizational impact by using Chi-square
17
19. The effect of the information quality measures
on user Satisfaction
Factors
Regression
coefficient
Standard
error
t-value P-value
Information accuracy 0.060 0.252 0.24 0.8116
Information timelines 0.502 0.285 1.76 0.0823
Information reliability 0.462 0.275 1.68 0.0975
Information up-to-datedness 0.252 0.265 0.95 0.3438
Decision supporting function 0.771 0.263 2.94 0.0045
19
20. The association of quality measures and information
satisfaction
Description
Information satisfaction
No In-between Yes Total Chisq p-
value
System
Quality
Ease of system use
No 1(25.0) 3(75.0) 0( 0.0) 4( 5.2) 18.0 0.002
In-between 0( 0.0) 19(59.4) 13(40.6) 32(41.6)
Yes 1( 2.4) 12(29.3) 28(68.3) 41(53.2)
Results
understanding
No 1(100.0
)
0( 0.0) 0( 0.0) 1( 1.3) 58.7 <.001
In-between 1( 3.2) 23(74.2) 7(22.6) 31(40.3)
Yes 0( 0.0) 11(24.4) 34(75.6) 45(58.4)
Terminology
understanding
No 1(33.3) 0( 0.0) 2(66.7) 3( 3.9) 32.7 <.001
In-between 1( 2.6) 27(69.2) 11(28.2) 39(50.6)
Yes 0( 0.0) 7(20.0) 28(80.0) 35(45.5) 20
21. Implementing Health Management
Information Systems: Measuring
Success in Korea’s Health Centers
Young Moon Chae, Suck Il Kim, Byung Hwa Lee,
Sung Hae Choi, In Sook Kim
(Published in International Journal of Health Planning and Management,
1994; 9: 341-8)
21
22. Introduction
• Background
– Quality of health statistics produced at the district level is still very low
so that health status of the district cannot be accurately determined.
– Management data needed for planning, organizing, and evaluating
health programs are not readily available for district managers.
– Productivity and morale of most health workers are low because they
suffer from heavy workloads of tedious paper works and simple
tabulation.
– Health centers and health subcenters are not responding to the
changing needs of primary health services, which has been changed
from family planning or MCH to chronic degenerative diseases
(e.g. hypertension, diabetes)
• Significances
– HMIS can be regarded as management innovation within health center
to meet the unrelenting challenge of providing and assuring health
services to the community.
– Relatively little research, however, has been done on measuring the
processes of introducing a management innovation into a health center.
22
23. Purposes
• To assess the extent to which a management innovation,
in this case a HMIS, is perceived as successful to health
center personnel
• Specifically,
– To examine the three key measures of successful implementation
of HMIS: productivity of health center staffs, adoption process, and
the satisfaction of visitors to the health center.
– To compare user satisfaction between computerized center and
not-computerized center
23
24. Methods
• Research design – longitudinal design with control
– Three surveys were conducted to measure the changes in
productivity and adoption process (knowledge, persuasion,
decision, implementation, and confirmation) of health workers
over time during the period of 20 months.
– The effects of the Health Management Information System
(HMIS) on the quality of services to the visitors were also
measured 7 months after the 3rd survey by comparing the
quality of services between the study health center and the
similar health center as a control group
24
27. Statistical Analysis
• The changes in three adoption variables (knowledge, persuasion,
and decision) were compared between the survey 1 and survey 2
(before and after the implementation of the HMIS) using Wilcoxson
signed-rank test.
• The changes in job processing time, which is a surrogate measure
for the productivity of health center staffs, between the three surveys
were compared using the Bonferroni method because there were
significant differences in variances among the three groups.
• Three satisfaction variables (waiting time, credibility, and
convenience) of visitors between the two health centers were
compared with Chi-square test, and average score of these
variables were compared with t-test (1 point for the dissatisfied and
3 point for the satisfied).
27
31. Association between Smoking, Screening, and Death
Caused by Cervical Cancer in Korean Women
Odongua N, Chae YM, Kim MR, Yun JE, Jee SH
(Published in Yonsei Medical Journal, 2007; 48(2):
192-200)
32. Introduction
Cervical Cancer: 2004
1. Worldwide
– 510.000 new cases
– 288.000 deaths
2. United States
– 10,520 new cases
– 3,900 deaths
3. Korea
– … new cases
– … deaths
Developed
countries
17.0%
Developing
countries
83.0%
Risk Factors:
•Failure to obtain regular Pap smear
test
•HPV infection
•Cigarette smoking
•Sex at an early age
•Multiple sexual partners
•Late menarche
•Early age at first delivery
•High parity
34. Proportion of Cancer Cases among
Korean Women (1999-2001)
stomach
16.4%
breast
13.7%
colorectal
10.4%
cervix
9.8%
lung
8.0%
liver
7.3%
thyroid
6.7%
gallbladder
3.5%
ovary
3.0%
pancreas
2.5%
others
18.7%
Source: National Cancer Center, Korea
35. Objectives
The purpose of this study is to explore and investigate the risk factors
for cervical cancer among Korean women from KCPS.
Specifically:
1. To estimate the age-adjusted standardized incidence and
mortality rates of cervical cancer among Korean women aged 30
years and older.
2. To investigate the relationship between cervical cancer and
reproductive factors.
3. To investigate the relationship between cervical cancer and
smoking and other behavioral factors.
40. Number of cervical cancer cases
by age group (n=2,520)
Incidence
405
263
145
98
34
0 318
283
121
0
226
330
235
21
41
0
200
400
600
800
1000
30-39 40-49 50-59 60-69 ≥70
Age group
Numberofcases
Women with abnormal result
Women with normal result
Unscreened women
41. Age-adjusted standard incidence rates
by age group
Incidence rate
13.6
12.6
10.6
7.9
1.3
13.6
12.2
9.0
7.4
1.1
0.0
9.0
7.6
4.5
0.80.0
31.1
18.1
13.6
2.5
0
10
20
30
40
30-39 40-49 50-59 60-69 ≥70
Age group
Rateper100,000
All women (overall rate 45.9)
Unscreened women (43.2)
Women with normal result (21.9)
Women with abnormal result (65.2)
42. Number of cervical cancer deaths
by age group (n=209)
Mortality
12
19 16 18 19
0
12
24
12 9
0
11
22
27
8
0
20
40
60
80
30-39 40-49 50-59 60-69 ≥70
Age group
Numberofdeaths
Women with abnormal result
Women with normal result
Unscreened women
43. Age-adjusted standard mortality rates
by age group
Mortality rate
0.4
0.7
0.9
1.0
0.5
0.4
0.9
1.0
1.4
0.6
0.0
0.3
0.6
0.4
0.4
0.0
1.5
1.2
1.6
0.5
0
0.5
1
1.5
2
30-39 40-49 50-59 60-69 ≥70
Age group
Rateper100,000
All women (overall rate 3.4)
Unscreened women (4.2)
Women with normal result (1.8)
Women with abnormal result (4.7)
49. Incidence and mortality rate of cervical
cancer among all women
Cervical cancer:
– Incidence rate: 45.9 per 100,000.
• The highest incidence rate: women aged 30-39 years old
– Mortality rate: 3.4 per 100,000
• The highest mortality rate: women aged 50 or older
Pap smear test:
– Women who had never been screened by Pap smears:
• 1.7-fold high risk of incidence for cervical cancer
• 2-fold high risk of mortality for cervical cancer
50. Discussion
• The Healthy People 2010 target for cervical cancer is a
reduction in mortality to 2.0 deaths per 100,000 women.
• Since 1998, the rate remains near 3.0 deaths per 100,000
women. The U.S. Preventive Services Task Force strongly
recommends screening for cervical cancer in women who
have been sexually active and have a cervix.
• Pap smear screening should be targeted for younger women.
As a result, we can also reduce the overall burden of cervical
cancer among Korean women.
• Future studies are needed.
51. Discussion
• This large prospective cohort study of Korean women showed
that several reproductive factors including age at menarche,
age at first delivery, menopause status and long-term
estrogen exposure were independent risk factors for
development of cervical cancer.
• Death due to cervical cancer among Korean women was
associated with high level of fasting serum glucose (FSG)
(RR=1.73, 95%CI=1.03-2.92), late stage of hypertension
(RR=2.00, 95%CI=1.06-3.59) and smoking.
52. Evaluation of Mobile Phone-based
Diet Game for Weight Control
Wonbok Lee, Young Moon Chae, Sukil Kim,,
Seung Hee Ho, Inyoung Choi
(Published in Journal of Telemedicine and Telehealth,
2009; 16: 270-275)
52
53. Introduction
• Background
– 3rd National Health and Nutrition Survey conducted in 2005 reported
that 32% of Korean adults were obese, based on the same criteria.
Obesity is related to a metabolic syndrome and therefore obese
people are at a greater risk of developing cardiovascular disease
– web-based therapy management system with mobile phone access
has been developed and distributed to obese patients to support
weight management.
• Significance
– We developed the SmartDiet, a mobile phone-based weight control
system that not only closely tracks an obese patient’s daily nutrition
intake, but also has games that users can play to learn about weight
control.
– The SmartDiet is different from the previous dieting applications in
a sense that users could download and implement personalized diet
ary information onto their personal mobile phones
53
54. Objectives
• This study evaluated the effectiveness of the mobile
phone application SmartDiet with respect to acquiring
dietary information, weight control, and user satisfaction.
• Specific objectives are:
– To describe demographic characteristics and lifestyle of study
population
– To analyze the effects of SmartDiet by comparing body
composition between case and control group
– To analyze user satisfaction
54
55. Research Design
• Subjects
– The effectiveness of the SmartDiet was evaluated
based on the pretest-posttest with control design.
– 19 subjects were assigned to the case or intervention
group, and 17 subjects were assigned to the control
group.
• Statistical Analysis
– Differences in demographic characteristics and
lifestyle between the intervention group and the
control group were analyzed by the Chi-square test.
– The changes in body composition before and after the
intervention for both groups were compared by the
paired t test.
55
59. Management Issues in Health Information
Systems
59
Kwak EA, Chae YM, Ho SH, Kim GK. Management issues in hospital
information systems.
(Published in Journal of Korea Society of Medical Informatics.
2007; 13 (1): 9-18)
60. • Purpose
– This study was conducted to identify management issues in
hospital information systems
– Specifically,
• To identify management issues in 2007 and compare them
with 1999
• Classify management issues by using McFarlan’s framework
• Methods
– Sample survey of 50 managers from 28 hospital information
centers.
– Two rounds of interview surveys were conducted by using the
Delphi method.
60
61. Changes in Management Issues
61
Key Issues
Rank 7-year
Change2007 1999
Top Management Support 1 1 0
Cooperation within IS Organization 2 4 +2
PACS 3 18 +15
Disaster Recovery 4 - NEW
Improving IS Strategic Planning 5 17 +12
Standardization 6 - NEW
Legalization of Electronic Medical Record (EMR) 7 6 -1
Improving Information Security & Control 8 3 -5
EMR 9 12 +3
Educating System Engineers 10 7 -3
Reference: Kwak EA, Chae YM, Ho SH, Kim GK. Management issues in hospital information
systems. Journal of Korea Society of Medical Informatics. 2007; 13 (1): 9-18
62. Classification of Management Issues Based on
McFarlan’s Framework
62
High
Present
Low
Key Operational Strategic
Educating System Engineers Top Management Support
EMR
Aligning the IS Organization within the
Enterprise
PACS Standardization
Disaster Recovery Improving IS Strategic Planning
Support High Potential
Improving Information Security & Control Integrated HIS
Making Effective Use of Data Resource IS/IT Strategy & Planning
Building Patient Information System Infrastructure for U-Health
Recruiting & Developing IS Human
Resource
Activation of Ubiquitous Application
System
Collaborative IT System Organization &
Hospital
Developing Laboratory Information
System (LIS)
Improving the Effectiveness of Software
Development
Low High
Future
63. Competency-based Learning for Distance
Education in System Analysis and Design
Byung Hwa Lee, Young Moon Chae, Tomiko Hokama,
Seok Kim
(Published in Asia-Pacific Journal of Public Health.
2010; 22(3): 299-309)
63
64. Purposes
• To assess how well SAD competencies could be taught
via a CBL-based e-learning course that we developed for
students registered at the ICUH.
• Specifically,
– To develop a CBL-based curriculum by linking various
competencies to the learning goals of SAD.
– To develop a CBL model for SAD based on the curriculum
– To measures the competency scores of students three times
(before, during, and after the class) by using a competency diary,
in order to assess the students’ progress.
– To identify the factors affecting learning effectiveness.
64
65. Research Framework for CBL Course Evaluation
65
Learning process
Case and
problem
Tutor’s role
Student’s role
Learning
effectiveness
Multiple regression
