1. Evaluation of Methods Handling Missing Diary Data
for Statistical Analysis in Dry Eye Studies
Hui-Chun T. Hsu, Dale Usner, Richard Abelson
Statistics & Data Corporation, Tempe, AZ.
thsu@sdcclinical.com
5319/D0080
Patient symptom diaries are a commonly used method to collect efficacy data in clinical
trials such as those for dry eye treatments. Typically, patients are asked to report the severity
of several ocular symptoms multiple times per day over the course of a study that may last
weeks or even months. Missing data are common, especially when the patients are asked
to complete many items or when diaries are collected over many time points. Along with
analyzing the observed data only, the missing observations can be imputed based on
other available data to obtain statistically plausible values. There are several possible ways
to handle the missing diary data, each of which will result in a different statistical outcome.
The goal of imputing missing diary data is to use available data to estimate the statistical
outcome that would have been achieved with no missing data.
Purpose
5000 sets of diary data were randomly created from a multivariate normal distribution for two
treatment groups (active and placebo). For each simulation, a complete two weeks of daily
diary data was generated for 50 subjects per treatment group, assuming a 0.6 treatment mean
difference on a scale of 0-5 with a standard deviation of 1 and a correlation of 0.85 between
consecutive diary days. Responses were provided at the subject level. Two percent of the
observations were randomly set as missing and ten percent of the subjects were randomly
selected as early withdrawals. Several imputation methods were used to handle the missing
data and compared: (1) last observation carried forward (LOCF); (2) baseline observation
carried forward (BOCF); (3) post-baseline worst observation carried forward (WOCF); (4)
subject mean; (5) treatment group mean. Table 1 presents an example of implementing the
listed methods.
A mixed model accounting for repeated measures within each subject was used for statistical
analysis. The percentages of times where the results indicated significant treatment differences
based on the different imputation methods were compared to the complete simulated data
as well as observed data only with Mixed Model Repeated Measures [(6), MMRM]. Further-
more, the concordance and discordance of significance between each imputation method
and the complete simulated data were summarized. Discordance includes two parts: false
positive (i.e., significant using imputation data but not significant using complete simulated
data), and false negative (i.e., not significant using imputation data but significant using
complete simulated data).
Methods
Compared to the analysis based on the complete simulated data, treatment group mean
imputation (5) yielded an artificially higher percentage of significance, whereas BOCF (2)
yielded an artificially lower percentage of significance. Both methods (2) and (5) showed
relatively lower concordance rates compared to the other methods. The discordance in
BOCF (2) was primarily a function of a higher false negative rate (5.7%) while the treatment
group mean imputation (5) was primarily a function of a higher false positive rate (2.3%). All
other methods had similar percentages of significance, concordance and discordance rates
as the analysis based on the complete simulated data.
Results
All of the above imputation methods, including analyzing the observed data only with MMRM,
arevalidformissingdatahandling.Morecompleximputationmethods(e.g.multipleimputation)
can also be used and will be included for comparison in future research. More than one
imputation method is recommended to apply to the clinical research for sensitivity analysis.
The methods yielding similar percentages of significance as analysis based on the complete
simulated data with high concordance rates are recommended.
Conclusion
Figure 1. Bar Chart of Significance Rate Comparison (Power)
Table 1. Implementation of different imputation methods on the data: assuming subject 1 is in active
treatment group (group mean = 2.4) and subject 2 is in placebo group (group mean = 3)
Figure 3. Bar Chart of Discordance Comparison
Figure 2. Bar Chart of Concordance Comparison