Medical University of South Carolina: Using Big Data and Predictive Analytics to Drive Clinical Quality Improvement Initiatives

1. Using Big Data and Predictive Analytics to Drive
Clinical Quality Improvement Initiatives:
A Case of Kidney Transplantation.
Titte R Srinivas, David J. Taber, Zemin Su, Jingwen Zhang, Girish Mour,
David Northrup, Arun Tripathi, Brendan J Fowkes, Justin E. Marsden,
William P. Moran, Patrick Mauldin
Medical University of South Carolina, Charleston
IBM Corporation, Armonk NY
Session ID: 1854

2. “A good hockey player plays where the
puck is.
“…A GREAT hockey player plays
where the puck is going to be.”

3. The challenge of quality….
n Safe
n Patient centered
n Timely
n Effective
n Efficient
n Equitable

4. Joint Principles of the Patient-Centered
Medical Home
AAFP,ACP, AOA, AAP March, 2007
n Personal physician
n Practice in teams
n Whole person orientation
n Care is coordinated and integrated
n Quality and safety are hallmarks
n Enhanced access

5. Background
• Predictive models in kidney transplantation derived from national data
(UNOS, SRTR) lack longitudinal patient level data, thereby limiting
accuracy.
• Adding patient level data capturing dynamic post-transplant clinical
evolution to predictive models, may improve predictive accuracy for graft
loss (GL) risk.
• Complete capture of patient level clinical data in real time would require an
approach that extracts, collates and curates both structured and
unstructured data from electronic health records (EHR)
• These large amounts of data are notable for volume, velocity, variety and,
verified veracity; An operational definition of Big Data
• As such analytic techniques would also need to handle such data

6. Predictive Diagnostics in Different Medical Contexts
1 - Specificity
Sensitivity
0
0.5
1
0 0.5 1
C=0.5Stroke: c = 0.87
AMI: c = 0.85
CABG: c = 0.74
Transplant: c = 0.653
• Less acuity
• Longer follow up window
• Competing Risk
• CURRENT MODELS DO NOT CAPTURE
LONGITUDINAL CLINICAL EVOLUTION
Why are Transplant Models Inferior?
Courtesy: JD Schold
(modified with
permission)

7. Attributes of the Ideal Predictive
Model for Graft Loss
• Appropriate to Center’s Population and customizable
• Ability to discriminate across levels of risk
• Feasibility of build around clinically actionable variables
• Uses data available within the EMR that are collected in the
context of standard patient care
• Biologically relevant to the extent of current understanding
including social determinants and care processes
• Ability to inform on individual patient trajectories and capture
dynamic longitudinal clinical evolution in the temporal context
of routine clinical care

8. Objectives
• Articulate an approach to capture longitudinal post-transplant
clinical evolution among kidney transplant recipients by
capturing structured and unstructured elements from the EHR
• Build predictive models for graft loss and mortality using
patient level data
• Compare model performance with those derived of national
data
• Deploy predictive models in a clinician facing interface
through the electronic medical record to drive post transplant
clinical care

9. Workflow of Data Extraction,
Storage, Analysis and Deployment
Watson
Natural
Language
Processing
Hadoop data
storage
Predictive analytic, Data
Processing and Scoring

10. IBM SPSS Modeler & C&DS
• Predictive modeling.
• Data integration and
processing
• Creation of dynamic
variables (e.g. eGFR
trajectory)
• Predictive model
building.
• Model deployment
and scoring
Automation. (C&DS:
Collaboration &
Deployment Services)

11. Watson Natural Language
Processing (NLP)
• Blood pressure
• Heart rate
• Temperature

12. eGFR: TRAJECTORY
Similar approaches can be used to incorporate hemoglobin, blood pressure and heart
rates into longitudinal models
Time to first Max
Standard Deviation
Slope from last Max
To day 90 (forced at Max)

13. Statistical Analysis
§ Risk models were developed for 1 & 3 year GL and 3 year
Mortality)
Each of these risk models incorporated variables as follows:
§ Model 1: OPTN/UNOS/SRTR variables
§ Model 2: UNOS + Tx Database Variables
§ Model 3: UNOS + Tx Database + EHR Comorbidities
§ Model 4: UNOS + Tx Database + NLP variables + Trajectory
variables

14. Statistical Analysis
• Backward selection in the Multivariable Firth
logistic model using an exit p-value at the 20
percent level for variable selection
• For verification of selection,
o Step-wise AIC variable selection criteria
o Lasso variable selection technique
• Clinical adjudication if discrepancies of variables
were revealed between methods
• Methods to account for over-fitting were utilized

15. Rationale for Variable Inclusion
Variable Source; Category
KDRI UNOS; Kidney Quality
Caregiver Status Transplant Database; Social
Determinant
Education Status Transplant Database; Social
Determinant
ICD-10 Comorbidities EHR; Comorbidities; Cardiometabolic
risk
Banff Lesion Scores EHR, NLP; Immunologic Risk
CMV, BKV PCRs EHR: Immunologic Risk
Cardiovascular events EHR; CV Risk; Access to care
Blood Pressures, Blood sugars EHR, NLP; Biology, Cardiometabolic
risk
Hemoglobin and eGFR
Slopes/trajectories
EHR; Biology of Kidney Function
Readmit Counts EHR; Processes of care, Access to
care

16. Rationale for Variable Inclusion
Variable Source; Category
KDRI UNOS; Kidney Quality
Caregiver Status Transplant Database; Social
Determinant
Education Status Transplant Database; Social
Determinant
ICD-10 Comorbidities EHR; Comorbidities; Cardiometabolic
risk
Banff Lesion Scores EHR, NLP; Immunologic Risk
CMV, BKV PCRs EHR: Immunologic Risk
Cardiovascular events EHR; Immunologic Risk
Blood Pressures, Blood sugars EHR, NLP; Biology, Cardiometabolic
risk
Hemoglobin and eGFR
Slopes/trajectories
EHR; Biology of Kidney Function
Readmit Counts EHR; Processes of care, Access to
care

17. Rationale for Variable Inclusion
Variable Source; Category
KDRI UNOS; Kidney Quality
Caregiver Status Transplant Database; Social
Determinant
Education Status Transplant Database; Social
Determinant
ICD-10 Comorbidities EHR; Comorbidities; Cardiometabolic
risk
Banff Lesion Scores EHR, NLP; Immunologic Risk
CMV, BKV PCRs EHR: Immunologic Risk
Cardiovascular events EHR; CV Risk, Access to care
Blood Pressures, Blood sugars EHR, NLP; Biology, Cardiometabolic
risk
Hemoglobin and eGFR
Slopes/trajectories
EHR; Biology of Kidney Function
Readmit Counts EHR; Processes of care, Access to
care

18. Rationale for Variable Inclusion
Variable Source; Category
KDRI UNOS; Kidney Quality
Caregiver Status Transplant Database; Social
Determinant
Education Status Transplant Database; Social
Determinant
ICD-10 Comorbidities EHR; Comorbidities; Cardiometabolic
risk
Banff Lesion Scores EHR, NLP; Immunologic Risk
CMV, BKV PCRs EHR: Immunologic Risk
Cardiovascular events EHR; CV Risk, Access to care
Blood Pressures, Blood sugars EHR, NLP; Biology, Cardiometabolic
risk
Hemoglobin and eGFR
Slopes/trajectories
EHR; Biology of Kidney Function
Readmit Counts EHR; Processes of care, Access to
care

19. Rationale for Variable Inclusion
Variable Source; Category
KDRI UNOS; Kidney Quality
Caregiver Status Transplant Database; Social
Determinant
Education Status Transplant Database; Social
Determinant
ICD-10 Comorbidities EHR; Comorbidities; Cardiometabolic
risk
Banff Lesion Scores EHR, NLP; Immunologic Risk
CMV, BKV PCRs EHR: Immunologic Risk
Cardiovascular events EHR; CV Risk, Access to care
Blood Pressures, Blood sugars EHR, NLP; Biology, Cardiometabolic
risk
Hemoglobin and eGFR
Slopes/trajectories
EHR; Biology of Kidney Function
Readmit Counts EHR; Processes of care, Access to
care

20. Rationale for Variable Inclusion
Variable Source; Category
KDRI UNOS; Kidney Quality
Caregiver Status Transplant Database; Social
Determinant
Education Status Transplant Database; Social
Determinant
ICD-10 Comorbidities EHR; Comorbidities; Cardiometabolic
risk
Banff Lesion Scores EHR, NLP; Immunologic Risk
CMV, BKV PCRs EHR: Immunologic Risk
Cardiovascular events EHR; CV Risk, Access to care
Blood Pressures, Blood sugars EHR, NLP; Biology, Cardiometabolic
risk
Hemoglobin and eGFR
Slopes/trajectories
EHR; Biology of Kidney Function
Readmit Counts EHR; Processes of care, Access to
care

21. Rationale for Variable Inclusion
Variable Source; Category
KDRI UNOS; Kidney Quality
Caregiver Status Transplant Database; Social
Determinant
Education Status Transplant Database; Social
Determinant
ICD-10 Comorbidities EHR; Comorbidities; Cardiometabolic
risk
Banff Lesion Scores EHR, NLP; Immunologic Risk
CMV, BKV PCRs EHR: Immunologic Risk
Cardiovascular events EHR; CV Risk, access to care
Blood Pressures, Blood sugars EHR, NLP; Biology, Cardiometabolic
risk
Hemoglobin and eGFR
Slopes/trajectories
EHR; Biology of Kidney Function
Readmit Counts EHR; Processes of care, Access to
care

22. Rationale for Variable Inclusion
Variable Source; Category
KDRI UNOS; Kidney Quality
Caregiver Status Transplant Database; Social
Determinant
Education Status Transplant Database; Social
Determinant
ICD-10 Comorbidities EHR; Comorbidities; Cardiometabolic
risk
Banff Lesion Scores EHR, NLP; Immunologic Risk
CMV, BKV PCRs EHR: Immunologic Risk
Cardiovascular events EHR; CV Risk, access to care
Blood Pressures, Blood sugars EHR, NLP; Biology, Cardiometabolic
risk
Hemoglobin and eGFR
Slopes/trajectories
EHR; Biology of Kidney Function
Readmit Counts EHR; Processes of care, Access to
care

24. 3 Year Graft Loss Risk and Data Sources
Odds Ratios; Logistic (Firth)
Odds
Ratio
95% Profile-Likelihood
Confidence Limits p≤0.05
Model 1: UNOS
KDRI 3.975 2.197 7.194 *
Age at Transplant 0.987 0.970 1.004
Female 0.588 0.360 0.936 *
Blood Type B 1.542 0.883 2.596
Model 2: UNOS + Transplant
Database
KDRI 4.160 2.290 7.566 *
Age at Transplant 0.985 0.968 1.002
Female 0.624 0.381 0.997 *
Primary Care Giver 0.469 0.295 0.756 *

25. 3 Year Graft Loss Risk and Data Sources
Odds Ratios; Logistic (Firth)
Odds
Ratio
95% Profile-Likelihood
Confidence Limits p≤0.05
Model 1: UNOS
KDRI 3.975 2.197 7.194 *
Age at Transplant 0.987 0.970 1.004
Female 0.588 0.360 0.936 *
Blood Type B 1.542 0.883 2.596
Model 2: UNOS + Transplant
Database
KDRI 4.160 2.290 7.566 *
Age at Transplant 0.985 0.968 1.002
Female 0.624 0.381 0.997 *
Primary Care Giver 0.469 0.295 0.756 *

26. 3 Year Graft Loss Risk and Data Sources
Model 3: UNOS + Transplant
Database + Comorbidity OR 95% CI p≤0.05
KDRI 4.239 2.330 7.734 *
Age at Transplant 0.985 0.968 1.002
Female 0.623 0.375 1.012
Primary Care Giver 0.492 0.307 0.797 *
Cerebrovascular Disease 0.248 0.027 0.978 *
Cardiac Arrhythmias 1.712 1.031 2.788 *
Alcohol Abuse 2.446 0.793 6.463
Depression 1.829 0.928 3.419

27. 3 Year Graft Loss Risk and Data Sources
Model 3: UNOS + Transplant
Database + Comorbidity OR 95% CI p≤0.05
KDRI 4.239 2.330 7.734 *
Age at Transplant 0.985 0.968 1.002
Female 0.623 0.375 1.012
Primary Care Giver 0.492 0.307 0.797 *
Cerebrovascular Disease 0.248 0.027 0.978 *
Cardiac Arrhythmias 1.712 1.031 2.788 *
Alcohol Abuse 2.446 0.793 6.463
Depression 1.829 0.928 3.419

28. 3 Year Graft Loss Risk and Data Sources
Model 4: UNOS + Tx Database +
Comorbidity +
Post-Transplant Trajectory OR 95% CI p≤0.05
KDRI_MED_Imputed 3.06 1.49 6.28 *
AgeAtTransplant 0.98 0.96 1.00 *
Female 0.57 0.32 0.99 *
Primarycaregiver 0.40 0.23 0.69 *
Cerebrovascular Disease 0.23 0.02 1.13
Obesity 0.66 0.38 1.11
Alcohol Abuse 3.22 0.89 9.78
Depression 1.91 0.86 3.98
Pulse Pressure_SDev_1yr 1.14 1.06 1.22 *
AcuteMI_comp_1yr 11.14 2.15 54.30 *
Cardiac_or_Vascular_Event_1yr 2.98 1.74 5.10 *
HGBSlope_7dto1yr_Imputed 0.72 0.56 0.87 *
Pulse_Mean_1yr 1.03 1.00 1.07 *
MaxEGFR_1yr 0.99 0.98 1.00 *
ED visit count_1yr 1.31 0.93 1.74
Acute_Banff_Score_Max_2_1yr 1.37 1.22 1.54 *

29. 3 Year Graft Loss Risk and Data Sources
Model 4: UNOS + Tx Database +
Comorbidity +
Post-Transplant Trajectory OR 95% CI p≤0.05
KDRI_MED_Imputed 3.06 1.49 6.28 *
AgeAtTransplant 0.98 0.96 1.00 *
Female 0.57 0.32 0.99 *
Primarycaregiver 0.40 0.23 0.69 *
Cerebrovascular Disease 0.23 0.02 1.13
Obesity 0.66 0.38 1.11
Alcohol Abuse 3.22 0.89 9.78
Depression 1.91 0.86 3.98
Pulse Pressure_SDev_1yr 1.14 1.06 1.22 *
AcuteMI_comp_1yr 11.14 2.15 54.30 *
Cardiac_or_Vascular_Event_1yr 2.98 1.74 5.10 *
HGBSlope_7dto1yr_Imputed 0.72 0.56 0.87 *
Pulse_Mean_1yr 1.03 1.00 1.07 *
MaxEGFR_1yr 0.99 0.98 1.00 *
ED visit count_1yr 1.31 0.93 1.74
Acute_Banff_Score_Max_2_1yr 1.37 1.22 1.54 *

30. 3 Year Graft Loss Risk and Data Sources
Model 4: UNOS + Tx Database +
Comorbidity +
Post-Transplant Trajectory OR 95% CI p≤0.05
KDRI_MED_Imputed 3.06 1.49 6.28 *
AgeAtTransplant 0.98 0.96 1.00 *
Female 0.57 0.32 0.99 *
Primarycaregiver 0.40 0.23 0.69 *
Cerebrovascular Disease 0.23 0.02 1.13
Obesity 0.66 0.38 1.11
Alcohol Abuse 3.22 0.89 9.78
Depression 1.91 0.86 3.98
Pulse Pressure_SDev_1yr 1.14 1.06 1.22 *
AcuteMI_comp_1yr 11.14 2.15 54.30 *
Cardiac_or_Vascular_Event_1yr 2.98 1.74 5.10 *
HGBSlope_7dto1yr_Imputed 0.72 0.56 0.87 *
Pulse_Mean_1yr 1.03 1.00 1.07 *
MaxEGFR_1yr 0.99 0.98 1.00 *
ED visit count_1yr 1.31 0.93 1.74
Acute_Banff_Score_Max_2_1yr 1.37 1.22 1.54 *

31. Effect of Layering Data Sources
on Model Performance

32. Model 1: UNOS
Model 2: UNOS + Transplant Database
Model 3: UNOS + Transplant Database + Comorbidity
Model 4: UNOS + Tx Database + Comorbidity +
Post-Transplant Trajectory

33. Pairwise Model Comparison
Model 1: UNOS only
Model 2: UNOS + Transplant Database
Model 3: UNOS + Transplant Databases + EHR Comorbidity
Model 4: UNOS + Transplant database + NLP + EHR Comorbidity + EHR Post-
Transplant Trajectory
1 Year Graft Loss 3 Year Graft Loss 3 Year Mortality
Data Model A vs Data Model B P-value Data Model A vs Data Model B P-value Data Model A vs Data Model B P-value
2 (AUC=0.741) vs 1 (AUC=0.716) 0.416 2 (AUC=0.665) vs 1 (AUC=0.661) 0.838 2 (AUC=0.765) vs 1 (AUC=0.765) 1.000
3 (AUC=0.769) vs 1 (AUC=0.716) 0.170 3 (AUC=0.712) vs 1 (AUC=0.661) 0.021 3 (AUC=0.768) vs 1 (AUC=0.765) 0.476
4 (AUC=0.873) vs 1 (AUC=0.716) <.001 4 (AUC=0.846) vs 1 (AUC=0.661) <.001 4 (AUC=0.838) vs 1 (AUC=0.765) 0.004
3 (AUC=0.769) vs 2 (AUC=0.741) 0.119 3 (AUC=0.712) vs 2 (AUC=0.665) 0.006 3 (AUC=0.768) vs 2 (AUC=0.765) 0.476
4 (AUC=0.873) vs 2 (AUC=0.741) <.001 4 (AUC=0.846) vs 2 (AUC=0.665) <.001 4 (AUC=0.838) vs 2 (AUC=0.765) 0.004
4 (AUC=0.873) vs 3 (AUC=0.769) 0.001 4 (AUC=0.846) vs 3 (AUC=0.712) <.001 4 (AUC=0.838) vs 3 (AUC=0.768) 0.007

34. Pairwise Model Comparison
Model 1: UNOS only
Model 2: UNOS + Transplant Database
Model 3: UNOS + Transplant Databases + EHR Comorbidity
Model 4: UNOS + Transplant database + NLP + EHR Comorbidity + EHR Post-
Transplant Trajectory
1 Year Graft Loss 3 Year Graft Loss 3 Year Mortality
Data Model A vs Data Model B P-value Data Model A vs Data Model B P-value Data Model A vs Data Model B P-value
2 (AUC=0.741) vs 1 (AUC=0.716) 0.416 2 (AUC=0.665) vs 1 (AUC=0.661) 0.838 2 (AUC=0.765) vs 1 (AUC=0.765) 1.000
3 (AUC=0.769) vs 1 (AUC=0.716) 0.170 3 (AUC=0.712) vs 1 (AUC=0.661) 0.021 3 (AUC=0.768) vs 1 (AUC=0.765) 0.476
4 (AUC=0.873) vs 1 (AUC=0.716) <.001 4 (AUC=0.846) vs 1 (AUC=0.661) <.001 4 (AUC=0.838) vs 1 (AUC=0.765) 0.004
3 (AUC=0.769) vs 2 (AUC=0.741) 0.119 3 (AUC=0.712) vs 2 (AUC=0.665) 0.006 3 (AUC=0.768) vs 2 (AUC=0.765) 0.476
4 (AUC=0.873) vs 2 (AUC=0.741) <.001 4 (AUC=0.846) vs 2 (AUC=0.665) <.001 4 (AUC=0.838) vs 2 (AUC=0.765) 0.004
4 (AUC=0.873) vs 3 (AUC=0.769) 0.001 4 (AUC=0.846) vs 3 (AUC=0.712) <.001 4 (AUC=0.838) vs 3 (AUC=0.768) 0.007

35. Mutable factors associated with
1 & 3 year graft loss and mortality
fall into 2 categories
Transplant associated risk
(historically managed by transplant
nephrology)
Non-Transplant associated risk
(historically managed by primary care)

36. Model Deployment in The Clinic

37. 37

38. FROM….. TO…..

39. Limitations
• This is a proof of concept which needs to
be evaluated in the clinic
• Single center study that needs to be
replicated in other centers and across
EMR and analytic platforms
• Rules of engagement relevant to real time
deployment in the clinical setting are not
defined

40. Conclusion
• A Big Data approach to curating diverse data sources adds
significant accuracy to 3 year graft loss prediction models,
and makes meaningful use of data in EHRs
• Our approach captures clinical complexity and longitudinal
dynamic evolution across several clinically mutable
domains.
• This solution is executable daily through EHR workflows
and could help optimize outcomes and deliver value
• This approach can be extended to most disease states using
the care processes that characterize clinical transplantation

41. Team
MUSC
• Titte Srinivas, MD
• David Taber, Pharm D
• Patrick Mauldin, PhD
• Jingwen Zhang, MS
• Zemin Su, MS
• Justin Marsden, MS
• William Moran, MD, MS
• David Northrup, MS
• Mark Daniels, MS
• Karthick Gourisankaran, MS
• Leslie Lenert, MD, MS
• Katie Reilly, BA
• Martha Sylvia, RN,MSN
IBM
• Haroon Anwar, MS
• Arun Tripathi, Ph.D.
• Salvatore Galascio, MS

42. Questions ?

43. Thank You !

45. MUSC Kidney Transplant Modeling
One year Graft Loss (GL)
(Transplant dates between 1/2007 and 6/2015)
• 1,176 patients analyzed
• 45 had 1yr GL (3.8%)
• Logistic Firth
• 90 days exposure period
• Backward Selection + Stepwise AIC
• AUC = 0.873 (0.807 – 0.939)
• Harrell’s Adjusted AUC = 0.829
• BCA (Bias Corrected and Accelerated)
Bootstrap AUC = 0.764 – 0.913

46. MUSC Kidney Transplant Modeling
Three year Graft Loss (GL)
(Transplant dates between 1/2007 and 6/2015)
• 891 patients analyzed
• 89 had 3yr GL (10.0%)
• Logistic Firth
• 1 year exposure
• Backward Selection + Stepwise AIC
• AUC = 0.846
• Harrell’s Adjusted AUC = 0.811
• BCA (Bias Corrected and Accelerated)
Bootstrap AUC = 0.791 – 0.869

47. MUSC Kidney Transplant Modeling
Three year Mortality (ML)
(Transplant dates between 1/2007 and 6/2015)
• 880 patients analyzed
• 76 had 3yr ML (8.9%)
• Logistic Firth
• 1 year exposure
• Backward Selection + Stepwise AIC
• AUC = 0.838
• Harrell’s Adjusted AUC = 0.800
• BCA (Bias Corrected and Accelerated)
Bootstrap AUC = 0.766 – 0.868

48. MUSC Kidney Transplant Modeling
(Transplant dates between 1/2007 and 6/2015)
• Firth Logistic Regression
• 90 days exposure period for 1 year
survival models and 1 year exposure for 3
year models
• Backward Selection + Stepwise AIC

49. Logistic (Firth) regression odds ratio and 95% confidence intervals (Model 4: UNOS + Velos + EHR Comorbidity +
EHR Post-Transplant Trajectory)
Odds Ratio (95% CI)
1 Year Graft Loss 3 Year Graft Loss 3 Year Mortality
UNOS
Age at Transplant 0.98 (0.96, 1.00)* 1.03 (1.00, 1.05)*
Female 0.57 (0.32, 0.99)*
African American 0.46 (0.26, 0.81)*
KDRI 2.48 (0.95, 6.43) 3.06 (1.49, 6.28)* 1.81 (0.88, 3.69)
Blood Type B 3.41 (1.55, 7.35)*
Waitlisting Time (Years) 0.76 (0.56, 0.98)*
First Week Dialysis 2.48 (1.20, 5.00)*
Diabetes 1.67 (0.93, 3.02)
Obesity 0.50 (0.23, 1.06) 0.66 (0.38, 1.11) 0.65 (0.36, 1.14)
Private Insurance 0.46 (0.21, 0.91)*
Previous Kidney Transplant 2.31 (0.99, 5.05)
Graft Loss by 1 Year 2.77 (1.13, 6.54)*
Transplant Database
Finish High School 0.47 (0.23, 0.97)*
Smoker 2.61 (0.87, 6.91)
Primary Care Giver Identified at Transplant 0.40 (0.23, 0.69)*
EHR***
Cerebrovascular Disease 0.07 (<0.01, 0.65)* 0.23 (0.02, 1.13)
Cardiac Arrhythmias 2.16 (1.01, 4.52)*
Alcohol Abuse 3.71 (0.87, 12.73) 3.22 (0.89, 9.78)
Drug Abuse 3.55 (0.63, 15.50)
Depression 1.91 (0.86, 3.98) 0.44 (0.14, 1.13)
Transplant LOS (Days) 1.08 (1.01, 1.26)*
Acute MI during Exposure** 11.14 (2.15, 54.30)*
Cardiac or Vascular Event during Exposure 2.48 (1.06, 5.66)* 2.98 (1.74, 5.10)* 2.23 (1.21, 4.08)*
BK>500 during Exposure 1.93 (0.90, 3.88)
CMV>500 during Exposure 0.20 (0.02, 0.88)*
Pulse Mean during Exposure 1.03 (1.00, 1.07)* 1.04 (1.00, 1.07)*
SBP Mean during Exposure 0.97 (0.94, 1.00)*
Pulse Pressure SDev during Exposure 1.14 (1.04, 1.25)* 1.14 (1.06, 1.22)*
Glucose Mean during Exposure 0.99 (0.98, 1.00)
HGB Slope perMonth - Day7 until End of Exposure 0.78 (0.58, 0.93)* 0.72 (0.56, 0.87)*
Max eGFR during Exposure 0.97 (0.95, 0.99)* 0.99 (0.98, 1.00)*
eGFR Slope from Max Value until End of Exposure 0.83 (0.74, 0.99)* 0.87 (0.73, 0.98)*
Tacrolimus_SDev_during Exposure 1.18 (0.97, 1.42)
Inpatient Readm Count during Exposure 1.42 (1.03, 1.93)* 1.16 (0.98, 1.37)
Emergency Department Visit Count during
Exposure 1.31 (0.93, 1.74)
NLP
Max Acute Banff Score during Exposure 1.37 (1.22, 1.54)*

50. Statistical Analysis
• Assess Overfit: To assess and adjust for overfitting, results
were validated using the Harrell Optimism Correction; The
Harrell optimism corrected c statistic is an “honest” estimate
of internal validity, penalizing for overfitting
• Internal Validation
o Bootstrapping with Bias Correction methodology were used for
model internal validation and AUCs for the ROC curve were
used to determine and compare model accuracy
o Empirical cross check between actual observed events and
model predicted risk scores was conducted by clinicians
• Assess Methodology: To assess the “robustness” of the
logistic methodology, models were also created using survival
analysis (Cox Proportional Hazard)

51. Supplemental Material: Cox proportional Hazard Model Results
Supplemental Table 1: One Year Graft Loss
Green: represents variables in Logistic Model but not in Cox Model
Yellow: represents variables in Cox Model but not in Logistic Model
Cox Proportional Hazard Model
(Concordance (C-index) = 0.873, se = 0.043)
Effect Hazard Ratio LHR ULR p-value
KDRI
Blood Type B 3.080 1.496 6.343 0.002
Obesity 0.588 0.288 1.202 0.145
Waiting Time (Years) 0.776 0.608 0.990 0.042
Finish High School 0.589 0.297 1.166 0.129
Smoker 2.688 1.086 6.655 0.033
Cerebrovascular Disease
Cardiac Arrhythmias 2.206 1.113 4.372 0.023
Alcohol Abuse 2.809 0.845 9.332 0.092
Drug Abuse
Cardiac or Vascular Event_90days 2.539 1.206 5.345 0.014
CMV>500_90days 0.128 0.017 0.959 0.046
SBP Mean_90days 0.974 0.949 1.000 0.048
Pulse Pressure SDev_90days 1.134 1.043 1.234 0.003
Glucose Mean_90days 0.985 0.972 0.997 0.018
HGB Slope perMonth from Day7 until 90days 0.814 0.753 0.879 <.0001
eGFR Slope from Max Value until 90days 0.820 0.724 0.932 0.002
Max eGFR_90days 0.965 0.946 0.985 0.001
Inpatient Readm Count_90days 1.282 0.977 1.682 0.073
Female 0.486 0.233 1.013 0.054
Age At Transplant 1.027 0.998 1.056 0.068
Married 0.501 0.243 1.032 0.061
Depression 0.382 0.101 1.444 0.156
HGB Mean from Day7 until 90days 0.738 0.542 1.004 0.053
Max Acute Banff Score_90days 1.151 0.992 1.334 0.063
Tacrolimus Mean_90days 0.827 0.656 1.042 0.107

52. Supplemental Table 2: Three Year Graft Loss
Cox Proportional Model
(Concordance (C-index) = 0.843, se = 0.031)
Effect Hazard Ratio LHR ULR p-value
KDRI 2.739 1.480 5.069 0.001
Age at Transplant 0.983 0.964 1.002 0.081
Female 0.546 0.333 0.896 0.017
Obesity 0.646 0.401 1.039 0.071
Primary Care Giver 0.581 0.361 0.935 0.025
Cerebrovascular Disease 0.081 0.008 0.798 0.031
Alcohol Abuse
Depression
Acute MI_1yr 4.746 1.537 14.652 0.007
Cardiac or Vascular Event_1yr 2.411 1.461 3.979 0.001
Pulse Pressure SDev_1yr 1.147 1.078 1.221 <.0001
HGB Slope perMonth from Day7 until 1yr 0.817 0.758 0.881 <.0001
Pulse Mean_1yr 1.027 0.998 1.056 0.067
Max eGFR_1yr 0.985 0.975 0.995 0.004
Max Acute Banff Score_1yr 1.296 1.194 1.407 <.0001
ED Readm Count_1yr 1.260 0.944 1.682 0.117
Diabetes 0.647 0.375 1.116 0.118
Receive Disability 0.676 0.413 1.108 0.120
Smoker 1.733 0.837 3.592 0.139
Cardiac Arrhythmias 1.764 1.076 2.894 0.025
BK>500_1yr 0.522 0.252 1.078 0.079
CMV>500_1yr 0.648 0.357 1.177 0.154
eGFR Slope from Max Value until 1yr 0.862 0.762 0.972 0.017
Inpatient Readm Count_1yr 1.152 1.000 1.326 0.049
Tacrolimus Mean_1yr 0.865 0.725 1.033 0.109
Tacrolimus SDev_1yr 1.144 0.956 1.368 0.141
Green: represents variables in Logistic Model but not in Cox Model
Yellow: represents variables in Cox Model but not in Logistic Model

53. Supplemental Table 3: Three Year Mortality
Cox Proportional Model
(Concordance (C-index) = 0.815, se = 0.033)
Effect Hazard Ratio LHR ULR p-value
KDRI
Age at Transplant 1.029 1.007 1.050 0.008
African American 0.649 0.393 1.074 0.093
Private Insurance 0.491 0.249 0.965 0.039
Obesity
First Week Dialysis
Diabetes 1.567 0.930 2.638 0.091
Previous Kidney Transplant 2.481 1.244 4.949 0.010
Graft Loss_1yr 2.363 1.126 4.957 0.023
Depression 0.447 0.177 1.125 0.087
Transplant LOS (Days) 1.049 1.014 1.086 0.006
Cardiac or Vascular Event_1yr 1.990 1.164 3.401 0.012
BK>500_1yr
Pulse Mean_1yr
eGFR Slope from Max Value until 1yr 0.850 0.774 0.932 0.001
Inpatient Readm Count_1yr 1.113 0.968 1.280 0.133
Tacrolimus SDev_1yr 1.166 0.990 1.373 0.066
Distance to MUSC (Miles) 1.001 1.000 1.003 0.155
Receive Disability 0.687 0.408 1.157 0.158
Smoker 1.832 0.808 4.151 0.147
Peripheral Vascular Disorders 1.521 0.784 2.950 0.215
CMV>500_1yr 1.673 0.948 2.951 0.076
SBP Mean_1yr 0.980 0.962 0.999 0.040
HGB Mean from Day7 until 1yr 0.783 0.643 0.954 0.015
HGB Slope perMonth from Day7 until 1yr 0.796 0.565 1.123 0.194
Green: represents variables in Logistic Model but not in Cox Model
Yellow: represents variables in Cox Model but not in Logistic Model

54. CMV
Rejection
BK
DM
Smoker
MI
Married
Smoker
College
Grad
Immunologic Cardiometabolic Psychosocial
Date BK PCR
1/27/16 2.33E4
12/23/15 Neg
11/22/15 Neg
Date Blood Sugar
1/27/16 185
12/23/15 109
11/22/15 93
Model Output Model Output