Evaluating your training - a pragmatic approach. The East of England experience. Lepley, Hockley & Kuhn
•Descargar como PPT, PDF•
1 recomendación•202 vistas
Presented at LILAC 2010
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a Evaluating your training - a pragmatic approach. The East of England experience. Lepley, Hockley & Kuhn
Similar a Evaluating your training - a pragmatic approach. The East of England experience. Lepley, Hockley & Kuhn (20)
Más de IL Group (CILIP Information Literacy Group)
Más de IL Group (CILIP Information Literacy Group) (20)
Último
Último (20)
Evaluating your training - a pragmatic approach. The East of England experience. Lepley, Hockley & Kuhn
- 1. Evaluating your training: a pragmatic approach the East of England experience Deborah Lepley, Isla Kuhn, Pauline Hockley
- 2. Why do we do what we do?
- 3. Herding cats… them time-poor not a single cohort (not even multiple cohorts!) variety of reasons for attending adult learners
- 5. Herding cats… us! 31 libraries across 6 counties autonomous but collaborative style and delivery of training sessions content of session might vary depending on group vs 1-2-1, trainer, IT….
- 6. We deliver A core set of searching skills / techniques: Boolean Keyword Searching Subject Headings / MeSH Truncation Explode / Major Descriptors Using a variety of electronic resources: National Library for Health (now Health Information Resources / NHS Evidence) Pubmed Databases (Medline, CINAHL, Embase, etc) Cochrane Library My Journals TRIP
- 7. What we wanted to find out… Do trainees: Use resources after training Use search techniques after training Do they retain learning?
- 8. What we did Built on our previous Essex study Developed pre- and post-training evaluations Pilot using paper surveys Transfer to Surveymonkey Region wide roll out
- 9. The surveys Pre-Training Completed by all trainees prior to training Assessed baseline knowledge and resource / technique use & awareness Post-Training Link emailed 6 weeks after training Follow up depended on individual trainers Same questions on resource & technique use Additional questions on literature searching & impact of training
- 10. Who came?
- 13. Use of resources
- 14. Use of search techniques
- 15. Have they used their new skills? Learning is retained through reinforcement 75% did a literature search after training AND 89% used Subject Headings
- 16. What was the impact of training? Had to provide an expert report on one of my patients – I used the search tools and techniques learned on the course updated treatment guides used the training most recently to look at evidence for brief intervention with problem drinkers I was able to complete a systematic review on mifepristone –mistoprostol interval in MTOP Consultant Junior Doctor Occupational Therapist GP
- 17. Would you recommend the training? 99% said YES “In fact I already have recommended it! I found it opened up facilities that I didn't know existed.” “Finding what evidence is available in an efficient way is such an important skill. Being able to be confident that if evidence is not found it is not because skills are lacking but that the evidence is not available is also important. This training helps start on that journey.” Junior Doctor A Junior Doctor B
- 18. So what? Hurrah! we’re making a valuable contribution to the daily work and CPD of these professionals!
- 19. So what now? Improve/update the pre- & post-training questionnaires MCQs or other tests? Personalised post-training questions? Use benefits of http://bit.ly/butwYz Review training sessions in light of changing resources and skills of learners Get more sessions translated into online versions and evaluate further
- 20. Thank you Deborah Lepley deborah.lepley@nhs.net Isla Kuhn ilk21@cam.ac.uk Pauline Hockley In absentia
Notas del editor
- all started with archie - tried to reduce the amount of duplication of effort by physicians by creating a library of systematic reviews of the literature which could give definitive statement about best evidence (+ therefore practice) in particular situation. but these things take time, and when there isn't a systematic review available, you still have to go out and see if anyone's solved the problem that you have already.... evidence based practice..... sackett..... evidence based practice - 5 steps Step 1 - Convert your information need into an answerable question Step 2 - Find the best evidence Step 3 - Appraise search results for validity and usefulness Step 4 - Apply the findings to your clinical practice and step 5 - evaluate your professional performance (step 6 - disseminate findings....) skills required have since become embedded in professional competencies - royal colleges, "modernising medical careers", "skills for health" etc etc.....
- - even with the european working time directive, staff are time poor - while at least there are regular intakes of new jnr doctors in august and feb (note to self - avoid hospital at these times!) (though not of other staff) they can't be considered a cohort in the way a new intake of students can. - skill levels vary enormously (predictable) - reasons for attending vary enormously (cpd, direct patient care, publishing article, professional exams, "on list of things to do and they've got a free afternoon - can I fit them in??" ) and sessions are likely yto be a one-off, so few/limited opportunities for follow-up - adult learners who often just want to know what they need to know right now, but who are also self-selecting (have recognised a rumsfeld moment - they know that they don't know something....)
- 562 pre – training surveys completed across the region 163 matched post-training surveys Response rate of 29%
- First of all, To try and ascertain whether trainees had retained learning we asked the same question on the pre and post surveys about awareness of electronic resources. For each resource we asked trainees to rate whether they had Never Heard of it Heard of it, but not used it Used it occaisonally Used it frequently Bearing in mind that training content would have varied across the region, and not all trainees would have been trained on all resources, the results were certainly encouraging. This slide shows the main resources and the numbers reporting they had never heard of it before and then 6 weeks after training. In all cases the numbers decreased, indicating less people had never heard of the resources after the training than before, Even if individuals did not then access resources after training at least there is some recall of fact, and we can hope that they would be aware of where to go for information if the need arises. The 2 there that still have around 50 people after training are EEL and Proquest. EEL is the Eastern Electronic Library for Health and Social Care, which is promoted as a starting point for accessing resources by some, but not all of the participating Trainers, and Proquest a journal collection that though many trainees may have accessed it may have been part of wider access to electronic journals.
- We asked a similar question for search techniques. We listed the techniques: Keyword Searching Subject Headings MeSH Boolean Truncation Major / Explode And asked trainees to rate whether they had Never Heard of it Heard of it but not used Used during a search Again numbers reporting to have never heard of a technique decreased after training, showing trainees are recalling facts six weeks after a training session.
- However, we wanted to show more than just our trainees could remember and recall resources and techniques, we wanted to show they were actually using what they had learned. As afterall, we know that for learning to be properly retained is has to be used.
- The same is true for frequent use of search techniques. Were techniques used after training? This graph shows that yes, they were. Numbers self reporting to have used a technique frequently after training increased. Useage and the 2/3 things remembered
- So far then, we have shown trainees remember what they have learned, abd have shown they are using techniques and resources. But what about specific usage. We know that learning is most effectviely retained through reinforcement. Much of the training would have been around doing a literture search in a database, such as Medline or Cinahl. How many trainees had actually done their own search after training. We found that three quarters had done a search, 95% either fully or partially planned the search first and 89% of those had actually used subject headings in that search, either fully or partially.. Bell paper – need to reinforce learning through doing. Have they been doing? Yes - % done a literature search and used subject headings
- Another aspect of the training we were particularly interested in was the impact of it on trainees work activities or studies. We found that most reported that it did have an impact, but in very general terms. Such as “Increased confidence and knowledge to use data bases and search for evidence” and “It has taught me how to do literature searches and look for journals which have been key to my work” However, some examples given were more specific as shown here. Some respodents could demonstrate that the training had a direct impact on their work, with a consultant being able to do his oen literautre searcj to inform an expert report, the junior doctor undertaking a systematic review, the GP doing a specific search to help with patient care, and the occupational therapist who was able to find the evidence to update treatment guides. We feel we hsve the baisis here for critical incident technique Critical incident technique How are they using it after training - impact
- All the graphs are going in the right direction, and we’re making a valuable, measurable difference to the skill set of these professionals
- Link to Methods to increase response to postal and electronic questionnaires. Edwards PJ, Roberts I, Clarke MJ, Diguiseppi C, Wentz R, Kwan I, Cooper R, Felix LM, Pratap S. Cochrane Database Syst Rev. 2009 Jul 8;(3):MR000008. Review. PMID: 19588449 [PubMed - indexed for MEDLINE]