Ovretveit implementation science research course 1day sept 11

1. Implementation research and scienceThis PPT and other resources from:http://public.me.com/johnovr 1 John Øvretveit, Director of Research, Professor of Health Innovation and Evaluation, Karolinska Institutet, Stockholm, Sweden 24/09/2011

2. Objectives – after the workshop I will be better able to: Explain what implementation science and research is Describe the elements of an implementation programme Explain the strengths and limitations of an implementation research study Plan an implementation study which is useful to the users of the study Know where to go for more details 24/09/2011 2

3. How - Learning process Use tool to analyse implementation programme(CESSiM) Using tool to assess implementation (REAIM) Assess quality of research using DAGU tool Plan implementation and or research using these tools Process – John explain, You try, We assess. Do you have Resources – outline, questions and cases? Johnstone, Ammentor, Helm (1,3,2) 3 ,

4. Timetable Introductions what is implementation science and research? you say “hallo” Example implementation programme Group case analysis on elements CASSiM Implementation assessment Group case analysis REAIM Assess quality of research DAGU Planning an implementation study – phases, your examples? Group exercise Guide to resources & Q&A 24/09/2011 4

5. Is implementation about evidence based practice? What is evidence based medicine, evidence based healthcare? Professionals using research to make better informed decisions in their practice Usually evaluations showing the most effective treatment for a patient But - can be evaluations of service delivery model (eg team based care) or health promotion intervention Implement = enable more practitioners to use a more effective method in their practice 24/09/2011 5

6. Implementation also about using improvement methods An example of an “improvement method”? PDSA testing: A quality team tests a change to reduce waiting times using a Plan Do Study Act cycle (eg follow ups optional) Flow process mapping A quality team maps patient pathway and reduces delays and errors Implementation not just about getting more people to use effective treatments 6

7. Definitions: Implementation science and research “Implementation research is the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice, and, hence, to improve the quality and effectiveness of health” Ecclesand Mittman, Implementation Science 2006. 24/09/2011 7

8. Implementation research Mostly describes, evaluates and explains an implementation in different real-life settings. 24/09/2011 8

9. Confusion warning: IS overlaps with: ‘‘translation research’’: ‘‘an effective translation of the new knowledge, mechanisms, and techniques generated by advances in basic science research into new approaches for prevention, diagnosis, and treatment of diseases essential for improving health’’ Many other related types of research Knowledge translation research Quality improvement research etc 24/09/2011 9

10. Say hallo to the person sitting next to you Name, title and where you work Are you doing implementation research or involved in an implementation project? What do you want to learn about implementation research and practice? Specific question 24/09/2011 10

11. Group rules Choose chairperson, scribe to report, cuckoo clock Chairperson says how much time left, when the group will summarise and agrees the task and questions Make room for everyone to contribute. Cell phones off! 24/09/2011 11

12. Part 1: The elements of an implementation programmeExample: implementing and maintaining a care pathway for the last days of life CESSiM Content? Define What was the “thing” implemented? Efficacious? Was it evidence-based: was the guideline for the care pathway based on evidence of effective care? Structure and systems What was the implementation structure: the people and groups at different levels who took actions to enable others to follow the guideline 24/09/2011 12

13. CESSiM - continued Strategy: phases/steps, actions, Were there phases or steps in the implementation (different things done at different times?) What were the implementation actions: what were the 5 main things done by these people and groups to enable others to follow the guideline? (by whom and when? implementation Methods: Did they use implementation methods – systematic ways to do enable change? 24/09/2011 13

14. Case . 24/09/2011 14

15. Points 1 Implementation is different from the change being implemented Before/after change – anti-biotic prophylaxis 1 hr before surgery Implementation actions – training, reminders, performance data collection and feedback 2 Efficacy X Implementation = Effectiveness 24/09/2011 15

16. What does “fidelity” mean? Exact “digital” copy of proven practice Key questions: For which interventions is fidelity essential For which is adaption needed Are some parts of the intervention adaptable and some not (“core”) Eg must copy the change difference But can implement it in different ways Or do you have to copy implementation strategy as well? 24/09/2011 16

17. Making a difference depends more on effective implementation than on an efficacious intervention Drugs don’t work if the patient does not take them Implementation is methods to enable patients to get, and take medicines in the way proven to be effective An efficacious communication training for physicians will not improve patient-physician communication if it is not implemented effectively. 24/09/2011 17

18. One Implementation assessment method Reach (% target population participated) Efficacy (outcome success rate if implemented as in original research) Adoption (proportion of settings, practices that adopt the intervention) Implementation (extent to which the intervention is implemented as intended in the real world) Maintenance (extent to which a program is sustained over time) 18

20. Is this a good method to use to assess implementation?

21. Are all parts needed for implementation study?24/09/2011 19

22. Case: Ammentorp& Kofoed2011 24/09/2011 20

23. 1 Effectiveness of treatment implementation is more important than efficacy of the improvement 2 Context factors affect each step 21

24. Use CASSiM and REAIM to explain Falls Guideline Failure (Helm et al 2006) Groups – read out loud the abstract. From abstract in 15 mins summarise Content, Efficacious, Structure and systems, Strategy, Implementation methods 15 mins summarise: Reach, Efficacy, Adoption, Implementation, Maintenance 15 mins: what is your explanation for figure 2 and 3? 22

25. Case Results: In the internal medicine ward, the target incidence of 6% was met for four of the 18 months in the intervention period. In the neurology ward, the inci- dence target of 11% was met in five months. Discussion: Barriers to change and enabling factors may only become apparent during the implementation process itself. A strongly perceived need to change daily practice, a simple guideline, the hospital board’s support, an understanding of local barriers, monitoring of outcomes, a locally tailored multifaceted implemen- tation strategy, and voluntarily cooperating nurses are no guarantees for success. 24/09/2011 23

26. .Points . 24/09/2011 24

27. Assessing the quality of a research study - DAGU Description Could we copy this implementation? Were the measures and data gathered the right ones (were resources used and saved measured?) Attribution What else could have explained the findings (how good was the design for the users questions)? Generalisation Does the study explain in which settings similar findings could be expected, and in which settings findings may be different? Use Is it presented in a way users can easily understand and will it help with users practical decisions? 25

28. Give 0-5 score for Helms et al 2006 study Description Could we copy this implementation? Were the measures and data gathered the right ones (were resources used and saved measured?) Attribution What else could have explained the findings (how good was the design for the users questions)? Generalisation Does the study explain in which settings similar findings could be expected, and in which settings findings may be different? Use Is it presented in a way users can easily understand and will it help with users practical decisions? 26

29. Plan research or an implementation Group for designing an implementation project or programme Group for planning a research study of an implementation List implementations you want to plan 24/09/2011 27

30. Groups Present timeline picture of main phases and data for monitoring Group work: Decide the customer for your plan – who is to use the design or the research when its done? Use CASSiM to define elements Use REAIM to define data needed to monitor or for research Or look at frameworks at end of Johns Outline paper and choose which is most suited to the users questions Use DAGU to decide how to improve your research design 24/09/2011 28

31. Points . 24/09/2011 29

32. . . 24/09/2011 30

33. 24/09/2011 31 Conclusions Each person write down and then share in the group: These were the main points… This was new or surprising, for me… The most useful idea for my work was… What I would like to find out more about…

34. DETAILS 24/09/2011 32

35. Resources Handouts Tools Web sites . . . 24/09/2011 33

36. Readings Overview and guide to the literature: Fixsen et al 2005 . . . 24/09/2011 34

37. Research assessment – DAGU or AGUDMP Attribution (extent to which intervention influenced the change in outcome data documented, & relative influence of components or synergistic influence) Generalise Used (Useful - matched to users questions – then gives information and presented to suit audience) Description (theory informed data collection) Measures valid Presentation 24/09/2011 35

38. 24/09/2011 36 Conclusions Each person write down and then share in the group: These were the main points… This was new or surprising, for me… The most useful idea for my work was… What I would like to find out more about…