1. The Cochrane Library: Train the Trainer for Health Librarians Canadian Cochrane Network & Centre Tamara Rader, MLIS Cochrane Musculoskeletal Group Institute of Population Health, University of Ottawa
2. Canadian Cochrane Network and Centre (CCNC) Funding Organizations Canadian Institutes of Health Research including 6 CIHR Institutes CADTH
10. Evidence-based Practice Evidence-based Decision Making Evidence/Information Expertise or experience Values, preferences, needs - from research - highest quality - past experience - own judgement - usually patient perspective - based on priorities in life, health beliefs Decision-making in health care
16. What do we need to manage the flow of information, keep it in context, and not be mislead by individual studies?
18. “ It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomized controlled trials”. Archie Cochrane, 1979
19. Collaboration’s Mission Statement The Cochrane Collaboration is a unique worldwide organization that aims to help people make well informed decisions about health care by preparing, maintaining and promoting the accessibility of systematic reviews of the effects of health care interventions. www.cochrane.org
21. What are the different review types? Type: Feature: Scoping Literature Review Narrative Question What is known about a topic What is known about a topic Broad Search strategy Unknown Unknown Not known Selection of studies Known, but post hoc Unknown Not known Inferences Thematic, makes comparisons Descriptive May be based on evidence Advantages Rapid appraisal, identifies research gaps Maps what is known Breadth, historical, phenomena
28. Cochrane Review - unique Grey & published, in all languages Systematic manual searches of key journals Computerized Databases Review of reference lists Consultation with experts Identify Studies Review for Relevance Not Relevant Relevant Reject Critical appraisal Extract Data Analyze Data Conclusions Plain language summary Updates
30. Review Submission If no review, registered title becomes available to others Guidelines for Protocol and Review Process Updating ~ 2 years 6 Months Title Registration Approval of title by Editorial Board Protocol Submission If no protocol, registered title becomes available to other interested reviewers 1.5 Years
47. Structure of the Cochrane Collaboration Cochrane Database of Systematic Reviews Database of Abstracts of Reviews of Effects Cochrane Central Register of Controlled Trials* Cochrane Methodology Register Health Technology Assessment Database NHS Economic Evaluation Database *Specialized registers are added into Cochrane Central Register of Controlled Trials (Central) The Cochrane Library
48. Structure of the Cochrane Collaboration Cochrane Database of Systematic Reviews Database of Abstracts of Reviews of Effects Cochrane Central Register of Controlled Trials* Cochrane Methodology Register Health Technology Assessment Database NHS Economic Evaluation Database Structure of the Cochrane Collaboration *Specialized registers are added into Cochrane Central Register of Controlled Trials (Central) The Cochrane Library
56. Cochrane Reviews - Improved Status Flags Carol Lefebvre. The Cochrane Library – an update , Wiley Sunrise Seminar. MLA, Honolulu, Hawaii, 18 May 2009.
59. Boolean Operators Aquatic excercises osteoarthritis Aquatic exercises osteoarthritis Aquatic exercises osteoarthritis Aquatic exercises osteoarthritis Aquatic exercises osteoarthritis Aquatic exercises AND OR NOT osteoarthritis
65. Start with PICO P atient/ P opulation I ntervention C omparison O utcome Margaret Sampson. The Cochrane Library , CHLA, Halifax, May 30, 2008. http://www.library.ns.ca/files/Cochranetraining_0.ppt
77. The Cochrane Library has many features that make finding the information you are after fast and accurate…
78. “ searching for all text schizo, drugs, atypical and antipsychotic. in All Fields in The Cochrane Database of Systematic Reviews " An Advanced Search of The Cochrane Library…
79. … Returns the following results: You can toggle between results listed for the different databases in The Cochrane Library here Use these links to restrict your search results to Reviews only, or Protocols (reviews in progress) only. Save or edit your search using these links.
104. Levels of evidence, variable quality Research users (consumers, health care professionals and policy makers) often poorly trained in critical appraisal skills
106. What is a Randomized Controlled Trial (RCT) Evans, Jennifer. Figure 1 Design of a Randomised Controlled Trial In: Epidemiology in Practice: Randomised Controlled Trials. J Comm Eye Health, 1998;11(26) 27. http://www.cehjournal.org/download/ceh_11_26_026.pdf
139. Henderson I. Craig, Wilcken N, Ghersi D, Davis N, Parker S, Carrick S. How to Read and Do a Cochrane Systematic Review . Leura V International Breast Cancer Conference, Sydney Australia, November 10-14, 2004. http://www.ctc.usyd.edu.au/cochrane/publications/presentations/CBCG%20-%20Leura%20V%20workshop%20November%202004.ppt Updates Draft Protocol peer review Draft review peer review Final Protocol Published on Cochrane Library Final review Published on Cochrane Library Perform review Identify studies, apply eligibility criteria, extract and analyse data Question registered
Who are you? Needs Experience with Cochrane and or SRs This will be posted on the ccnc website and clug website.
Over 20,000 health journals published per year Variable relevance To keep current, GPs must read 19 new scientific articles per day (19 x 1 hr critical appraisal…19 hours per day!) (Davidoff et al BMJ 1995) Average time professionals have available to read = <1 hour/week Pure volume of individual studies Want a summary bottom-line. One alternative is the generation of evidence-based summaries, which are in turn based on the evidence generated from systematic reviews.
Influence informed decision-making *Ensure relevance to end users Some people think that EB medicine is just this (circled part), but it is all of this – the balance of information, expertise and values.
This was Archie Cochrane, an epidemiologist who was the originator of The Collaboration. This idea was based on his belief that there was a need for the medical community to gather together the best available evidence to ensure that best clinical practice was being used.
International, not for profit
So we’ve covered many of the studies that we had in our heirarchy the other choices were the narrative reviews and systematic reviews. Reviews are another source that have become more and more useful to the health care community. Basically a review is just that – an overview of the studies that have been done about a topic or question. The value of the review is that there are so many studies now published that it is hard to read all the papers and studies out there in a particular field – so this makes it more efficient for the health care community to keep up to date on the latest research. It has been estimated that a physician would need to read 17 articles a day to keep abreast of all the new resaerch in a particular area! There is also often many studies that have confusing or contradictory results and so the review can put all the studies together and come up with a final ‘what is known’ statement. So reviews can be quite useful to consumers and physicians who are generally to busy or don’t have the resources to find all the studies about a topic, read them and combine the results, etc. Unfortunately not all reviews are created equal and there has been some concerns about reviews in general. Any ideas about why some reviews may not be so well done. Some reviews cover so much information about a topic that it is not clear what the review is actually about and what conclusions can be drawn. Some reviews haven’t found all the studies about a topic and only included for examples studies that have been published in scholarly journals – some studies may not have been published but contain very valuable information. Other reviews may have found studies and are comparing studies that use oranges and other studies that use apples or comparing studies that were done very well or not done very well or rct’s to case control studies and giving them equal weight. And still other reviews may only include studies that the author feels supports his personal viewpoint and exclude studies that refute his personal beliefs. And this leads to why the Cochrane Collaboration created the systematic review and process.
Generic, rigorous method to answer questions Provides the ‘big picture’ synthesis of existing research
Dealing with rationale for the review it self…
Paper based scientific journals tend to limit the number of tables and figures which emphasises the need for clear and concise writing. The Cochrane Database of Systematic Reviews allows all of the components of the systematic review to be presented in a structured way which ensures the highest quality possible.
Compared to systematic reviews published in Medical Journals, Cochrane reviews have… Inclusiveness of perspectives…Cochrane reviews are careful not to impose their own values, preferences, and local context Multistage process All Cochrane reviews follow the same format No word limit Many journals contain word limits which forces the authors to condense very complex information Search strategies used to identify studies include non-English language and unpublished studies Provided with plain-language summaries for non-experts Multinational editorial teams Involved right from the start of the project try to ensure review applies to different parts of the World Updated every two years Keep audience in mind while writing: abstract, plain language summary There’s no reason why some of these principles can’t be applied to the syntheses that you do, if you aren’t already.
Cochrane is not the only organization to produce SRs. But just a few words to highlight how Cochrane reviews are unique from other systematic reviews and why we are seen as the ‘gold standard’: Broad scope of literature included – grey lit, hand searching, registries of trials Critical appraisal – each study in the review is quality assessed using validated tools Plain language summary Updated and maintained every ~2-3 years
Learn a lot about the literature…
Use of the 'Cochrane Inside' logo A form of endorsement suitable for material containing or derived from Cochrane Collaboration outputs is the 'Cochrane Inside' logo. People or organisations wishing to use this logo should follow the same procedure as outlined in this document, with the additional criterion as to whether the item to be endorsed with the 'Cochrane Inside' logo contains or substantially derives from Cochrane Collaboration output'. Where this criterion is not met, use of the ‘Cochrane Inside’ logo will be denied. Unlike the ‘Cochrane’ logo, the ‘Cochrane Inside’ logo will not be available for use from the Cochrane website, but should be requested from the Secretariat ( [email_address] ).
Just like any library, the Cochrane Library is a collection of resources. It is the main output of the Cochrane Collaboration and aims to bring together in one place reliable information about the effects of health care treatments or interventions. In the Cochrane Library health care is broadly defined, so in addition to many hundreds of medical conditions, you’ll find information on such diverse topics as injury prevention for pedestrians and cyclists, St John’s wort for depression, programs to reduce juvenile delinquency, the prevention of jet lag, advice on low-fat diets for obesity, and prayer for the alleviation of ill health.
As of Issue 2, 2007, this database also includes the methodology reviews that were previously part of the Cochrane Database of Methodology Reviews (CDMR; Methods Reviews). As of Issue 4 2008, this database includes systematic reviews of diagnostic test accuracy.
THIS IS COMPARED WITH SEARCHING THE CDSR ON PUBMED
CENTRAL includes details of published articles taken from bibliographic databases (notably MEDLINE and EMBASE), and other published and unpublished sources. CENTRAL records include the title of the article, information on where it was published (bibliographic details) and, in many cases, a summary of the article. They do not contain the full text of the article. About three-fifths of the records in CENTRAL are taken from MEDLINE. In addition, each Cochrane Review Group maintains and updates a collection of controlled trials relevant to its own area of interest, these are called ‘Specialized Registers’. Each group may also collect items which are not relevant to its own field of interest; these are known as ‘Handsearch Results’. The registers and handsearch results are assembled and collated by Wiley before incorporation into The Cochrane Library . All Review Groups’ specialised registers, the handsearch results register, relevant records retrieved from MEDLINE, and relevant records retrieved from EMBASE, are merged as described here . EMBASE records can be identified by a sole EMBASE ID in the Accession Number field.
Cochrane Central Register of Controlled Trials (Clinical Trials). Includes details of published articles taken from bibliographic databases and other published resources. CENTRAL records include the title of the article, information on where it was published and, in many cases, the abstract
For parallel group trials, the features of interest in a standard ‘Risk of bias’ table of a Cochrane review are sequence generation, allocation sequence concealment, blinding, incomplete outcome data, selective outcome reporting and other potential sources of bias.
Advanced Search is more flexible…
You can use NOT but it will automatically translate the languge to “AND NOT” in The Cochrane Library Osteoarthritis of the hip OR Osteoarthritis or the ,knee
Advanced Search is more flexible…
An even more effective way to search in order to obtain a more precise set of records, rather than having to wade through huge amounts of irrelevant records in order to find the ones of interest, is to develop a search strategy using the PICO model.
Evidence-based searching begins with PICO: before you start to search, think about what you’re really asking and what you’re really looking for This is the step where you start to formulate your question To compare the effectiveness and safety of aquatic-exercise interventions in the treatment of knee and hip osteoarthritis. Examples: Patients with knee or hip osteoarthritis Aquatic exercise interventions ? May be several??? Is it safe and effective No time period indicated
To compare the effectiveness and safety of aquatic-exercise interventions in the treatment of knee and hip osteoarthritis.
During this talk about Boolean AND, OR, NOT
Codes for each individual language
Need to identify what your question is… Acknowledge there are different types of questions out there…
For example: What is the best treatment for head lice? Does “kangaroo mother care” benefit premature babies?
Not for ‘on going’ research.
Virtual training resources and training tips training tips for different sources within the NeLH Cochrane Library: training materials, speakers’ notes and teaching tips
Excellent for searching details. Demonstrates almost all of the main features.
Small groups IF TIME
References also have been put on individual slides.
Strict conflict of interest guidelines
Depends on what you want
Select participants high-risk for outcome (high incidence) Likely to benefit and not be harmed Likely to adhere Measure baseline variables Randomize Eliminates baseline confounding Types (simple, stratified, block)
A second diagram from a production point of view.
Steering Group All Collaborative Review Groups, Methods Groups, Fields, the Consumer Network, and Centres are eligible to vote in the election of members to the Steering Group decisions are guided by goals and objectives set out in the Collaboration's Strategic Plan (http://www.cochrane.org/admin/stratplan.htm). The CCSG is elected to develop policies and strategies for the Collaboration. It has several sub-groups responsible for specific tasks. The Steering Group is supported by the Collaboration Secretariat. Secretariat Secretariat is currently based in the UK.
Centres The work of Collaborative Review Groups, Methods Groups, Fields, and the Consumer Network is facilitated in a variety of ways by the work of a dozen Cochrane Centres around the world (Australia, Brazil, Canada, China, Denmark, Germany, Italy, the Netherlands, South Africa, Spain, UK, and the USA).
They share responsibility for helping to co-ordinate and support members of the Collaboration in areas such as training, and they promote the objectives of the Collaboration at national level.
Collaborative Review Groups The main work of The Cochrane Collaboration is done by fifty two Collaborative Review Groups, within which Cochrane Reviews are prepared and maintained. The members of these groups, researchers, healthcare professionals, people using the health care services (consumers), and others, have come together because they share an interest in generating reliable, up-to-date evidence relevant to the prevention, treatment and rehabilitation of particular health problems or groups of problems. To become part of The Cochrane Collaboration, each Collaborative Review Group is required to prepare a plan outlining how it will contribute to the Collaboration's objectives. This describes who will have responsibility for planning, co-ordinating, and mentoring the Group's work (a co-ordinating editor supported by an editorial team). It also describes how the Group will identify and assemble in a specialized register as high a proportion as possible of all the studies relevant to its declared scope; and who, drawing on the studies in this register, will take responsibility for preparing and maintaining which reviews. Every Group appoints an individual to organise and manage the day-to-day activities of the Group, a review group coordinator.
More than 11,000 volunteer review authors produce the majority of the Cochrane Reviews through the Review Groups Of the 52 review groups in the collaborative, Canada hosts 5 of them. Within Canada over the past decade and more – many have recognized the value if the Cochrane work, and have joined in to assist: there are 17 Universities committed to the Centre’s work and one regional health Authority on board. 22 affiliated orgainzations. It is catching on!
Fields Fields focus on dimensions of health care other than health problems, such as the setting of care (e.g. primary care), the type of consumer (e.g. children), or the type of intervention (e.g. vaccines). People associated with Fields search specialise sources for relevant studies, help to ensure that priorities and perspectives in their sphere of interest are reflected in the work of Collaborative Review Groups, compile specialized databases, co-ordinate activities with relevant agencies outside the Collaboration, and comment on systematic reviews relating to their particular area.
Methods Groups For example, the Statistical Methods Group is assessing ways of handling different kinds of data for statistical synthesis, and the Applicability and Recommendations Methods Group is exploring important questions about drawing conclusions regarding implications for practice, based on the results of reviews.
Consumer Network The Cochrane Consumer Network provides information and a forum for networking among consumers involved in the Collaboration, and a liaison point for consumer groups around the world.
Located at the 16 health sciences centres across the country, plus 2 additional sites - Recruit and support people new members of Cochrane - Promote local and regional awareness
A systematic review should be as transparent as possible and include as much of the above information as the publication medium allows.
<Intervention> for <Health problem> Epinephrine for bronchiolitis <Intervention A> versus <Intervention B> for <Health problem> Oral rehydration therapy versus intravenous therapy for gastroenteritis <Intervention> for <Health problem> in <Participant group/location> Inhaled nitric oxide for respiratory failure in preterm infants
With Cochrane, you are working with Review groups right from the beginning Search strategy
Once you’re title has been accepted…
How does bias come into play?
Inclusion Criteria Types of studies Types of participants Types of interventions Types of outcome measures Search methods for identification of studies Electronic searches (R) Searching other resources (R) Grey literature (O) Handsearching (O) Reference lists (O) Correspondence (O) Data collection Selection of studies (R) Data extraction and management (R) Assessment of risk of bias in included studies (R) Data Analysis Measures of treatment effect (R) Unit of analysis issues (R) Dealing with missing data (R) Assessment of heterogeneity (R) Assessment of reporting biases (R) Data synthesis (R) Subgroup analysis and investigation of heterogeneity (R) Sensitivity analysis (R)
Cochrane does not allow commercial funding
Once your protocol has been published in the Cochrane Library…. Some people work on the review even before the protocol is published to get started as turnaround times vary from CRG to CRG.
All reviews in the Cochrane Library will have this format ‘ Plain language summary’ (Synopsis) - Summarize in style understood by consumers - Accessible on the internet (stand-alone document) 2 parts 1. Restate title: (256 characters) plain language stating participants, intervention and outcome 2. Body: (< 400 words) Importance : background, signs/symptoms, prevalence, intervention, rationale Main findings : easily understood numerical results, Adverse effects , Limitations ABSTRACTS 400 words Available on-line as stand-alone documents Target healthcare decision makers Structured Consistent with text, data, conclusions
e.g. the Appendices the exact search strings for all the databases for the search strategy for the review could go here
To ensure this, Cochrane authors grant the Collaboration world-wide licences for these activities, and do not sign over exclusive copyright to any journal or other publisher. A journal is free to request a non-exclusive copyright that permits it to publish and re-publish a review, but this cannot restrict the publication of the review by The Cochrane Collaboration in whatever form the Collaboration feels appropriate. To republish material published in the Cochrane Database of Systematic Reviews elsewhere, most particularly in print journals, authors must complete a ‘permission to publish’ form available in the Cochrane Manual ( www.cochrane.org/admin/manual.htm ), along with an explanation of the procedures to follow. Authors are strongly discouraged from publishing Cochrane reviews in journals before they are ready for publication in CDSR.
However, journals will sometimes insist that the publication of the review in CDSR should not precede publication in print. When this is the case, authors should submit a review for publication in the journal after agreement from their CRG editor and before publication in CDSR. Publication in print should not be subject to lengthy production times, and authors should not unduly delay publication of a Cochrane review either because of delays from a journal or in order to resubmit their review to another journal. To ensure this, Cochrane authors grant the Collaboration world-wide licences for these activities, and do not sign over exclusive copyright to any journal or other publisher. When this is the case, authors should submit a review for publication in the journal after agreement from their CRG editor and before publication in CDSR. Publication in print should not be subject to lengthy production times, and authors should not unduly delay publication of a Cochrane review either because of delays from a journal or in order to resubmit their review to another journal. Journals can also request revision of a review for editorial or content reasons. External peer review provided by journals may enhance the value of the review and should be welcomed. Authors are also encouraged to add the following statement to versions of Cochrane reviews that are published in journals: ‘ This paper is based on a Cochrane review first published [or most recently substantively amended, as appropriate] in The Cochrane Library YYYY, Issue X (see www.thecochranelibrary.com for information). Cochrane reviews are regularly updated as new evidence emerges and in response to comments and criticisms, and The Cochrane Library should be consulted for the most recent version of the review.’ The following modification of the disclaimer published in The Cochrane Library should be added to Cochrane reviews published in journals.
A new group formed within Cochrane to address this – coming up with new info/guidelines When registering a review with the Cochrane Collaboration, authors agree to keep it up-todate. This entails repeating, at periodic intervals, the steps involved in the original review. Some of the steps will require minimal effort (e.g. reviewing the research question to make sure it is still relevant) while others may require a substantial investment of time. Occasionally, review authors may decide to include a new analysis strategy in their updated review; for example, using statistical methods not previously available in RevMan. In general, new analysis strategies will represent substantive changes that merit editorial critique through the CRG's established editorial process. How often reviews need updating will vary depending on the production of valid new research evidence. Review authors should work with their editorial team to establish guides addressing when new research evidence is substantive enough to warrant a major update or amendment. The dates of such amendments must be recorded in the What’s New section of the review. It is Collaboration policy that reviews should either be updated within two years or should have a commentary added to explain why this is done less frequently.
If newer researchers are interested in conduvting a systematic review, they may collaborate and assist current reviewers with the update or take over the review with permission from the authors.
CRG websites or methods guidelines
Diagnostic tests aim to reduce uncertainty about an individual's condition and aid in the diagnosis and detection of disease. A plethora of tests are available for almost every condition imaginable. Many tests exist to detect the same condition and new tests are being developed all the time. A perfect test would identify all patients with the target condition, without making mistakes, but perfect tests are rare and the users of a test wish to know how well the test discriminates between individuals who have the target condition and those who have not, also known as the diagnostic test accuracy. The accuracy of diagnostic tests is studied by comparing the results of the test under evaluation with the results of a test known to be very good (the reference standard) in the same patients/participants. Clinicians, policymakers and patients routinely face a range of questions regarding diagnostic tests. They want to know if testing improves outcome, would like to know what test to use, to purchase or to recommend in practice guidelines, and how to interpret the results of testing. Systematic reviews can help practitioners and decision makers in answering these questions, by summarising the available evidence and helping to explain differences among studies on the same question. As with elsewhere in science, systematic reviews and meta-analyses can be used to obtain more precise estimates, when small studies addressing the same test and study participant types in the same setting are summarised. This combination of studies needs to be done in as reliable a way as possible. For this reason, it needs to be systematic. Systematic reviews of test accuracy use a predefined and explicit methodology. Accuracy measures the ability of the test to distinguish between persons with and without the target condition. Good accuracy is desirable but is not the only information required to assess the effectiveness of a diagnostic test. To show that one test does more good than harm in terms of patient outcomes, one may require randomized controlled trials of test and treatment strategies.
Some registers have been coded in detail: For example: - Type of intervention (e.g. chemotherapy, endotherapy) - Study type (RCT or CCT) The RCT’s and CCT’s are kept and made available for use in systematic reviews Specifically designed searches can be performed on the register by the trial search coordinator, that will retrieve references relevant to your systematic review question This resource is available to all reviewers by contacting the Trial Search Coordinator for the applicable CRG
Each group may also collect items which are not relevant to its own field of interest; these are known as ‘Handsearch Results’. The registers and handsearch results are assembled and collated by Wiley before incorporation into the Cochrane Central Register of Controlled Trials in The Cochrane Library . We often handsearch the last five years of one or two key conference proceedings in the subject area of the review.
Determining the focus of the review by defining the question is one of the most important steps in conducting a systematic review Guides the rest of the review from developing the search strategies; selecting studies; identifying the data to be extracted; analysing variation among results Allows potential readers to judge whether the review is interesting or relevant to their issues or interests Where do questions come from? Questions arise constantly from routine clinical practice; the introduction of new treatments or diagnostic tests should always lead to the question “ are they better than what we already have”; also get ideas from future research or future directions sections of existing reviews Taking the time to define the question through consultation with co-reviewers and the your Cochrane Review Group will pay dividends by avoiding wasted time and effort in conducting your review and will increase the chance that the review is scientifically and clinically meaningful
Focus on the best available evidence – for addressing questions regarding therapeutic efficacy is a randomized controlled trial For other questions, for examples studies of diagnostic accuracy tests, cohort studies may be more appropriate or realistic This is something you should consider up front – what study designs are likely to provide reliable data with which to answer their questions Main problem with including non RCT is that they are at greater risk of bias than RCT particularly the lack of control of unknown or unmeasured confounders