1. Goud R et al. Effect of guideline based computerised decision support on decision making of multidisciplinary teams: cluster randomised trial in cardiac rehabilitation. BMJ. 2009;338:b1440. CATCH-IT Presentation Andrew Cheng 1
2. Outline Background Cardiac rehabilitation Decision support systems CARDSS Trial design Results Limitations Follow up study Implications Future study and other research 2
3. Cardiac Rehabilitation Care provided after hospitalization for a cardiac incident or intervention Outpatient care (approx 6-12 weeks) Goals: improving physical condition regaining emotional balance reintegration (e.g. work resumption) patient education secondary prevention through lifestyle changes 3
4. Cardiac Rehabilitation in the Netherlands Approx 100 outpatient clinics 26,000 patients/year Large variation in referral patterns organization of treatment decision making Dutch Cardiac Rehabilitation Guidelines (2004) Needs assessment procedure 4 types of therapy: exercise, education, relaxation, lifestyle change 4
5. Cardiac Rehabilitation in the Netherlands Multidisciplinary teams Physical therapists Nurses Psychologists Dieticians Social workers Rehabilitation specialists Cardiologists 5
6. Clinical Decision Support Computer system designed to assist physicians and other healthcare professionals in clinical decision making to enhance patient care Levels of CDS Documentation (EMR) Basic alerts (drug-drug interactions) Patient-specific recommendations 6
8. CARDSS Cardiac Rehabilitation Decision Support System Needs assessment procedure from guidelines assessment of rehab goals and therapies developed with the GASTON framework supports structured information gathering e.g. QoLquestionnaire System workflow integration Includes own electronic patient record (EPR) No cardiac rehab in NL used an information system Therefore, host system and database had to be included Additional functionalities linked to record 8
9. CARDSS Studies 0. Pilot Study (2003) R. Goud, N. Peek, A.M. Strijbis, P.A. de Clercq, A. Hasman, A computer-based guideline implementation system for cardiac rehabilitation screening, Comput. Cardiol. 32 (2005) 323–326. Effect of CARDSS on guideline adherence (2005) Cluster randomized trial CARDSS and barriers to implementation Follow up qualitative study CARDSS with feedback and outreach visits (2008-?) Ongoing cluster randomized trial 9
10. Interest First study to my knowledge to evaluate the effect of CDS on decision making in teams For others: Health care settings with multidisciplinary teams interested in adding CDS to their center Health informaticians 10
11. CARDSS Architecture CDSS clinical decision support system Host system Patient information management system (PIMS) Database Clinical information CDSS-related information 11
13. 13 The decision tree heart rehabilitation Risk Behaviour Head Questions Personal Info There is talk of risk behaviour? Question 1: Smoked the patient for prerecording in the hospital?
14. Questionnaire Form 14 1) You has the last how frequently frustrated himself two weeks, impatiently or has irritated felt? Always Never 2) You has the last how frequently felt himself two weeks worthless or to a little able?
16. Pilot Study Prototype version of CARDSS Started 2003, 2 months 4 outpatient clinics volunteered, 134 patients No control group System was quickly accepted by its users and easily integrated into clinical workflows Several adjustments and additional functions were added to system 16
17. CARDSS Study Hypothesis: Care providers are more likely to adhere to clinical practice guidelines when they receive guideline-based decision support by an electronic system Trial registration Current Controlled Trials ISRCTN36656997 17
20. Trial Design Cluster (center) randomization Intervention group: CARDSS with CDS Control group: CARDSS without CDS Duration: 6 months Started 2005 Last center completed in July 2006 20
21. Outcome measure Guideline adherence by care providers 4 types of therapy 2 Standard: exercise, education 2 New: relaxation, lifestyle change 21
23. Results: Exercise Therapy 23 Overtreatment Undertreatment Control (No CDS) Treatment (CDS) Adherence: 84.7% Adherence: 92.6% Significant change in adherence
24. Results: Education Therapy 24 Control (No CDS) Treatment (CDS) Adherence: 63.9% Adherence: 87.6% Significant change in adherence
25. Results: Relaxation Therapy 25 Control (No CDS) Treatment (CDS) Adherence: 34.1% Adherence: 59.6% Significant change in adherence
26. Results: Lifestyle Change Therapy 26 Control (No CDS) Treatment (CDS) Adherence: 54.1% Adherence: 57.4% No significant change in adherence
27. Conclusions CARDSS improved adherence to guideline recommendations with respect to exercise, education, and relaxation therapy No effect for lifestyle change therapy Majority of clinics did not have therapy program available Considerable undertreatment of patients Many patients did not receive the treatment they were suppose to according to guidelines A lot of variation in adherence between clinics 27
28. Limitations Required motivated multidisciplinary teams in centers with adequate IT infrastructure System cannot enforce changes requiring organizational change High attrition rate from implementing and learning how to use new system CARDSS comes with own EPR Authors also led development of the CDS system 28
29. Limitations Possible Hawthorne or checklist effect Insufficient statistical power 21 centers less than the calculated 36 necessary centers Not enough information on how the adjusted difference values and CI were calculated No explanation on how the covariates affect adherence 29
30. Ethical Issues Ethics approval stated as not needed according to the medical ethics committee of the Academic Medical Centre in Amsterdam 30
31. Questions for the authors Why not collect baseline adherence data? What is the reason for large variation in adherence between centers? Why was ethics approval not needed? 31
32. Questions for the authors What effect did the initial learning curve to the system have on users’ performance and adherence to guidelines? How large of a role if any did usability of CARDSS play? How was patient compliance accounted for given its large influence on guideline adherence? 32
33. Questions for the authours Why were non-CARDSS buyers not interested in the system? Specialized rehab centers less eager to implement CARDSS since they already developed own protocol more detailed than national guidelines CARDSS was not interoperable with other information systems Future versions will support interoperability 33
34. Follow Up Qualitative Study Research Questions: What are the main barriers to implementation of the national guideline for cardiac rehabilitation? Which barriers were reduced by CARDSS? 34
35. Study Design Semi-structured interviews with users of CARDSS nurses, physiotherapists 21 participating clinics Same 21 in the first study 29 interviews 35
36. Results CARDSS improved implementation of guidelines when: Professionals were unfamiliar with the details of the guidelines CARDSS made it easier to follow the guideline e.g. calculation and interpretation of QoLscores Patients sometimes refused to undergo treatment e.g. willingness to participate in psychosocial therapy increased 36
37. Results CARDSS did not improve implementation of guidelines when: Environmental barriers existed e.g. lack of facilities, no reimbursement, or believed there was no reimbursement Conflicts with other departments were present e.g. exercise test from another department required prior to needs assessment procedure Organizational change needed 37
38. Conclusions CDS systems can provide advice at the point of care The knowledge base can be based on practice guidelines Improves adherence if barriers are related to knowledge or complexity of the guidelines 38
39. Implications CDS when implemented properly may improve adherence to clinical guidelines and help standardize care in multidisciplinary teams If CDS can be provided through the resident EPR already in use, one of the barriers to implementation is removed Change management strategies is required for barriers relating to workflow or organization changes 39
40. Third Follow Up Study Clustered randomized trial Ongoing Aim: Standardize the work processes in heart rehabilitation centers Stage 1: Jan 2008, duration 6 months? Stage 2: Late 2008, duration 15 months? 40
41. Principal Authours 41 Dr. Niels Peek project LEADER CARDSS-project University professor Academically medical centre Amsterdam Department clinical information science Dr. Rick Goud project executant CARDSS-project Research scientist Academically medical centre Amsterdam Department clinical information science
42. Other Research Subjective usability of the CARDSS guideline-based decision support system Studies in health technology and informatics Goud R, Jaspers MW, Hasman A, Peek N. Subjective usability of the CARDSS guideline-based decision support system. Stud Health Technol Inform. 2008;136:193-8. Investigate subjective usability of a guideline-based CDSS for outpatient cardiac rehabilitation Questionnaire to 68 professionals from 28 outpatient clinics Professionals who managed to smoothly integrate the system with their daily routine were more satisfied with ease of system use 42
43. Other Research Development of a guideline-based decision support system with explanation facilities for outpatient therapy. Computer methods and programs in biomedicine Goud R, Hasman A, Peek N. Development of a guideline-based decision support system with explanation facilities for outpatient therapy. Comput Methods Programs Biomed. 2008 Aug;91(2):145-53. Design considerations CARDSS architecture Functionalities of CARDSS 43
inferencing mechanism (usually a set of rules derived from the experts and evidence-based medicine)inference engine is a computer program that tries to derive answers from a knowledge base. It is the "brain" that expert systems use to reason about the information in the knowledge base for the ultimate purpose of formulating new conclusions
GASTON state of the art framework for building DSS, consists of an ontology-based guideline representation language, a guideline-modelling tool that enables guideline authors to formally describe and easily modify practice guidelines visually, and a guideline execution engineThe designers of the GASTON framework were willing to provide personal assistance in the development of CARDSS.
CARDSS consists of three different components, namely a CDSS, a host system, and a database. We will refer to the host system as patient information management system (PIMS).
The PIMS operates as front-end application that cardiac rehabilitation professionals use to conduct the needs assessment procedure and decide on cardiac rehabilitation therapies for their patients
Did the patient smoke prior to the admission to the hospital
How frequently in the last two weeks have you felt frustrated, impatient, or irritated?How frequently in the last two weeks have you felt worthless?
Public title, scientific title, acronym, study hypothesis, study design, country of recruitment
5 control arm centres discontinued their participation. 3 were reluctant to continue participation as they believed that the benefits of CARDSS without CDS did not compensate for the increased workload of learning to work with the system. 1 had to stop participation because of a temporary lack of personnel, and another centre accidentally deleted its CARDSS database during an update of the server’s operating system in the last month of the trial.
IT: 4 centers unable to join study duringenrollment period since they were unable to update infrastructure in time. No information system was used in rehab centers prior to CARDSS.Organizational change: lifestyle change therapyAttrition:3 excluded from intervention due to not recording decisions into CARDSS , 3 discontinued from control due finding it not worth the extra work in implementing CARDSS without CDS1 from each grp for too much missing data from initial data auditResearch: however they tried to reduce bias by blinding investigators during allocation procedure, use of objective measures, external evaluator and statistician.
Stats: Calculations showed that with a 6 month follow-up, 36 centres needed to detect a 10% absolute difference in adherence with 80% power at a type I error risk (α) of 5%Covariates: age, sex, diagnosis, weekly vol of new pts, whether center is specialized rehab or part of academic hospital
However, if barriers are related to workflow or organizational barriers, change management stratsreq
