This is a lecture I wrote to introduce my students to the concept of Evidence Based medicine. Goes hand in hand with many handouts, such as the parachute study. Special thanks to Dr. Brian Bledsoes lecture on EBM, from wich I pirated liberally.

1. Evidence Based EMS
2013 Advanced EMT Course

2. Definitions

3. Definitions: Research
• Research: noun ri-ˈsərch, ˈrē-ˈ
• 1 : careful or diligent search
• 2 : studious inquiry or examination;
• especially : investigation or
experimentation aimed at the discovery
and interpretation of facts, revision of
accepted theories or laws in the light of
new facts, or practical application of
such new or revised theories or laws
• 3 : the collecting of information
about a particular subject

4. Definitions: Science
• Science: noun ˈsī-ən(t)s
• 1 : the state of knowing : knowledge as distinguished from ignorance or
misunderstanding
• 2 a department of systematized knowledge as an object of
study <the science of theology>
• 3 knowledge or a system of knowledge covering general truths or the
operation of general laws especially as obtained and tested through scientific
method
• b such knowledge or such a system of knowledge concerned with the physical world and
its phenomena - i.e. natural science
• 4 : a system or method reconciling practical ends with scientific laws
• “cooking is both a science and an art”

5. “Science is neither a philosophy nor a belief system. It is
a combination of mental operations that has become
increasingly the habit of educated peoples, a culture of
illuminations hit upon by a fortunate turn of history that
yielded the most effective way of learning about the real
world ever conceived.”
Edward O. Wilson
Consilience: The Unity of
Knowledge
So then, what is Science?

6. Definition: Evidence
• Evidence: noun ˈe-və-dən(t)s, -və-ˈden(t)s
• 1 : an outward sign
• 2 : something that furnishes proof :
• specifically :something legally submitted to a tribunal to ascertain the truth of
a matter
• 3: to be seen

7. Definitions: Evidence Based Medicine
(EBM)
• “ “…The conscientious, explicit and judicious use of current best
evidence in making decisions about the care of individual patients.”
• Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS (January
1996). "Evidence based medicine: what it is and what it isn't". BMJ 312 (7023): 71–
2.PMC 2349778. PMID 8555924.
• “…The use of mathematical estimates of the risk of benefit and harm,
derived from high-quality research on population samples, to inform
clinical decision-making in the diagnosis, investigation or
management of individual patients.“
• Greenhalgh, Trisha. How To Read a Paper: The Basics of Evidence-Based Medicine. Wiley-
Blackwell, fourth edition, 2010, p. 1.

8. Background: Why EBM?
• Prior to the 1970’s, it was assumed that basic medical education,
clinical updates, and interaction with other providers was adequate to
guide good decision making.
• Research in the 1970’s found HUGE variance on how providers would
treat similar clinical conditions.
• Most patient care was based on tradition and/or experience rather than
evidence and research

9. Background: Why EBM?
• Most patient care was based on tradition and/or experience rather
than evidence and research
• “That’s the way we always did it, why change”
• “Practicing at the level of graduation”
• Institute of Medicine (IOM) Report-1985 Estimate: Only 15% of
medical practices based upon solid evidence.
• Probably much much less for EMS

10. BENCH TEST vs. BEDSIDE TEST

11. EBM: What it is…
• A means to take individual biases and common cognitive errors out of
the equation
• A means to keep up to date and evolve as the art and science of
medicine evolves.
• A means to apply a rigorous review to what we previously just
accepted as truth.

12. EBM: What it isnt
• “Cook-book” medicine
• Tends to require more thinking to apply it correctly
• “Cost-cutting” medicine
• Sometimes the cost of care will increase, though the effectiveness will also
(More bang for the buck)
• A replacement for clinical expertise
• A replacement for common sense
• Absolute rules and regulations that leave no room for deviation

13. KEY POINT:
• External clinical evidence can inform, but can never replace,
individual clinical expertise…..

14. EBM:
• The practice of evidence based medicine means integrating
individual clinical expertise with the best available external clinical
evidence from systematic research.
• “Individual Clinical Expertise”: What does this mean?
• The proficiency and judgment that individual clinicians acquire through
clinical experience and clinical practice
• Typically acquired over years (decades?) of actual practice….
• “Best Available Clinical Evidence”: What does this mean?
• clinically relevant research, often from the basic sciences of medicine, but
especially from patient centered clinical

15. EBM: Focus
• Focuses on:
• Accuracy of diagnostic tests including the clinical examination.
• Example: 80-70-60 Rule of blood pressure, 12 rhythm interpretation
• The power of prognostic markers,
• Example: Glascow Coma Scale, Trauma levels, Mechanism of Injury /Co-Morbid factors
• The efficacy and safety of therapeutic, rehabilitative, and preventive
regimens.
• MAST Pants
• Intubation

16. What does this mean to me?
• Good Medics/EMTs use both individual clinical expertise and the best
available external evidence, and neither alone is enough.
• Without clinical expertise, protocols risks becoming tyrannised by
evidence, becoming Cook Book medicine.
• Even excellent external evidence may be inapplicable to or
inappropriate for an individual patient.
• Without current best evidence, practice risks becoming rapidly out of
date, to the detriment of patients

17. Art or Science?
• Some health practices are more
about art than science.
• Chiropractic, for example, has
little science and a lot of art.
• Neurology today has a great deal
of science and little art.
• Have you ever met a neurosurgeon
with personality?

18. Art or Science?
• Neurology today has a great deal
of science and little art.

19. Is EMS Art or Science?
• The knowledge of EMS is science.
• The way it is applied is art.
• Excellent EMTs know the science
of EMS and use the art of EMS to
apply the science.

20. “…Medicine is an Art, masquerading as a science”
Robert. S. Cole, NR-P, OCD, Resident Weirdo

21. Art or Science?
• When you have a life threatening
illness or injury, would you seek
out a health care provider whose
practice was more art or more
science?

22. Art or Science?
• EMS is the safety net of society.
• We are often the first to provide
care to the injured and the infirm.
• Our practices must be based on
science.

23. But we must apply them with art!

24. Understanding the Research……

25. Understanding the Research
• EBM is based n research…
• To appropriately apply EBM, you must understand the research it is
based on, so you can apply it correctly…..
• Most research is based on the “Scientific Method”

26. The Scientific Method

27. The Scientific Method
• The great advantage of the scientific method is that it is unprejudiced.
• Scientific research is reproducible
• One does not have to believe a given researcher, one can redo the experiment and
determine whether his or her results are true or false.
• Scientific Research is unbiased
• The conclusions will hold irrespective of the state of mind, or the religious persuasion, or
the state of consciousness of the investigator and/or the subject of the investigation.
• It focuses on the results of the science, regardless of what they show.
• Scientific Research evolves as our understanding evolves
• It assumes that its conclusions may one day be disproven by new science, and that is OK
(after all , that is science)

28. Observation
Hypothesis
Predictions
Theory/ Empirical Law
Experiment

29. The Scientific Method: The Theory/
Empiracal Law
• A theory is accepted not based on the prestige or convincing powers
of the proponent, but on the results obtained through observations
and/or experiments which anyone can reproduce.
• In other words: the theory is not based on the credentials of the
scientists, rather on the strength and reproducibility of the theory
itself…

30. The Scientific Method: Observation
• Observation: During a discussion you and your coworkers bring up the idea that
there are more psychiatric emergencies when the moon is full.

31. The Scientific Method: Predictions
• Prediction: You predict that there will be more EMS calls for psychiatric
emergencies when the moon is full as compared to other times of the month.

32. The Scientific Method: The Experiment
• You need a study group and a control group…

33. The Scientific Method: The Hypothosis
• Hypothesis: Psychiatric emergencies are more common when the lunar cycle is in
the full moon phase.

34. The Scientific Method: Experiment
• Testing: You set up an experimental protocol to test your hypothesis.
• You need:
• A study method
• A study period
• Study Parameters
• A study group
• A control group

35. The Scientific Method: Experiment
• A study method
• You decide that you will review charts (AKA a retrospective review)
• A study period
• You decide to review the charts for a period of a year
• Study Parameters
• You will review all charts over the period
• You will define the term “Psychiatric Emergency”
• You will gather information over the course of the year that includes the date of all psychiatric
emergency patients.
• A study group
• Al patients that occur in your “Full Moon Period”
• Define “full Moon Period” as 2 days before and after a full moon.
• A control group
• All other patient contacts outside of your full moon period….

36. The Scientific Method: The Experiment
Results:
Full Moon Period
Days=65
Other Period
Days = 300
Total
Days = 365
Ψ patients = 10 Ψ patients = 74 Total Ψ patients = 84
Total Patients =
2,105
Total Patients = 9,300 Total Patients = 11,315

37. The Scientific Method: The Experiment
• Use of the scientific method in EMS.
• Results: Your study finds:
• During full moon period 0.5% of patients were Ψ patients.
• During other period 0.8% of patients were Ψ patients.
• During the entire study period 0.7% of patients were Ψ patients.

38. The Scientific Method: The Experiment
• Use of the scientific method in EMS.
• Results: Your study finds:
• The incidence of Ψ patients during the full moon period was 0.15 patients per day.
• The incidence of Ψ patients during the other moon phases was 0.25 patients per
day.
• The incidence of Ψ patients overall was 0.23 patients per day.
• What does this mean?????
• Psych patients are less common during the full moon.
• HYPOTHESIS DISSPROVED!
• What Your Hypothesis didn’t work out? That’s ok, because its SCIENCE, and the result is what counts , not
our assumptions going into the experiment….
• SO , now what do you do?

39. Answer: You change your hypothesis to
match your results
(unless you are a drug company, then you do it the other way
around….)
• Revise hypothesis:
• New hypothesis: Psychiatric emergencies are no more frequent during full
moon lunar phases than during other lunar phases.
• Based on research, this HYPOTHESIS is now ACCURATE (until disproven)

40. Now what do you do with your new and
proven/researched hypothesis?
• 1- TELL PEOPLE SO THEY CAN BENEFIT!
• Report findings. Paper should be detailed enough that anybody can follow it
and repeat your experiment in their own enviroment
• 2- Repeat study to determine whether findings can be repeated.
• i.e. does your study hold true in other systems?
• Are there any weakness in your study you can correct, and then avoid in a
new study?

41. EMS Research
• The more valid a study, the closer
it is to the truth!

42. EMS Research
• Bias:
• A mental leaning or inclination; partiality; prejudice; bent.
• The more a study design adheres to the scientific method, the less chance for
bias to affect the outcomes.

43. EMS Research
• Bias (sometimes flat deception)
has been a common practice in
medicine and EMS.
• Quackery and bias still permeates
many aspects of modern
healthcare.

44. EMS Research
• Validity:
• Whether the study measures what it was supposed to measure.
• Validity refers to the appropriateness of the interpretation of the results of a study.
• Constant:
• A characteristic or condition that is the same for all individuals in a study.
• Variable:
• A characteristic that takes on different values or conditions for different individuals.
• Dependent Variable: the variable being affected or assumed to be affected my the
independent variable.
• Independent Variables: the variables that affects (or is assumed to affect) the dependent
variable under study.
• Experimental Variable: at least one independent variable being manipulated by the
researcher.

45. Understanding the “evidence”

46. Levels of Evidence

47. Levels of Evidence

48. Levels of Evidence (pre 2010)
• American Heart Association
1. Positive randomized controlled trials.
2. Neutral randomized controlled trials.
3. Prospective, non-randomized controlled trials.
4. Retrospective, non-randomized controlled trials
5. Case series (no control group)
6. Animal studies
7. Extrapolations
8. Rational conjecture (common sense)

49. Levels of Evidence
• Used to be 7 (or 8  ) different ILCOR
levels of Evidence (LOE).
• In 2010 ILCOR reduced these to 5.
• The AHA has broken these into three
broad categories
• Level A – Highest standard of evidence.
• Level B – Most common.
• Level C – Anecdotal case reports,
consensus opinions, retrospective studies,
small studies, previous standard of care
without evidence to contrary

50. KEY POINT:
“ A recommendation with Level of Evidence B or C does not imply that the
recommendation is weak.
Many important clinical questions addressed in the guidelines do not lend
themselves to clinical trials. Even though randomized trials are not
available, there may be a very clear clinical consensus that a particular test
or therapy is useful or effective.”
-2010 AHA ECC Guidelines
Circulation 2010;122;S657-S664

51. Classes of Recommendations
• Most previous classes retained with better clarifications and descriptions.
•“Class Indeterminate” recommendations, which were used in 2005, are not
included in the 2010 AHA Guidelines for CPR and ECC. The elimination of the
term “Class Indeterminate” is consistent with the ACCF–AHA Classes of
Recommendation.
•When the AHA writing groups felt that the evidence was insufficient to offer a
recommendation either for or against the use of a drug or intervention, no
recommendation was given.

53. Types of Studies: Randomized Controlled
Trials
• Randomized Controlled Trial (RCT): Gold Standard
• A group of patients are randomized into an experimental group and a control
group. These groups are followed up for the variables/outcomes of interest.
• RCTs most closely approximate the scientific method and are the most valid of
studies.
• A “double blind study” a type of RCT is one in which neither the patient nor
the physician knows whether the patient is receiving the treatment of interest
or the control treatment.

54. Randomized Controlled Trial

55. Double Blind Study

56. Quasi-Experimental Studies
• Quasi-experimental studies use intact groups of subjects rather than
assigning subjects to groups at random.

57. Quasi-Experimental Studies
• Non-randomized controlled trial:
• 1 group receives intervention
• 1 group receives no-intervention (control)
• Subjects assigned to groups by methods other than randomization.

58. Cohort Studies
• A Cohort Study is a study in which patients who presently have a
certain condition and/or receive a particular treatment are followed
over time and compared with another group who are not affected by
the condition under investigation.

59. Cohort Study

60. Case-Control Studies
• Case Control Studies: Case control studies are studies in which
patients who already have a certain condition are compared with
people who do not.

61. Case-Control Studies

62. Case Series/Report
• A case series is a report on series of patients with an outcome of
interest.
• No control group is involved.
•
• “This one time, at band camp, I had a bunch of patients) with ABC
condition, we did XYZ treatment, and here is what happened as a
result. Aint it cool!”
• A case report is a narrative report of a single interesting case.

63. Historical Studies
• Historical studies are systematized searches for the facts and then
using the information to describe, analyze, and interpret the past.

64. Systematic Reviews
• A summary of the medical literature that uses explicit methods to
systematically search, critically appraise, and synthesize the world
literature on a specific issue.

65. Meta-Analysis
• A meta-analysis is a systematic review of multiple studies that have
already been done, rather than conducting a new study, to draw
conclusions based on the whole of these studies…..

66. Meta-Analysis

67. Meta-Analysis
• Meta-analysis of RCTs represents the highest level of scientific
evidence.
• Based on the theory that multiple RCT’s will show more truth than a single
RCT will….
• Allows for more objective appraisal of the evidence.
• Reduces the possibility of false negative or false positive results.

68. Meta-Analysis (Observational Studies)
• Less valid than RCTs.
• Still valid in that it gives better information of the population as a
whole.
• Most common means of studying EMS related interventions.
• Many EMS interventions cannot be tested with an RCT as it might be
unethical to withhold care from the control group.

69. Meta-Analysis (Observational Studies)
• Conclusions:
• 2 out of 3 trauma patients transported from the scene to a trauma center
have minor injuries based on validated trauma scoring criteria.
• 1 out of 4 patients transported are discharged from the emergency
department.

70. Animal Studies
• Animal studies provide insight into biology, particularly into specific
systems.
• For ethical reasons, some studies cannot be carried out on humans.
But can be carried out on animals
• Much resuscitation research conducted on pigs and dogs.
• Much toxicology and neurological research conducted on mice.
• Computer modeling are replacing many animal studies.

71. Expert Opinion
• Expert opinions can take various forms:
• Systematic reviews
• Narrative reviews
• Pure opinion pieces
• Position Papers produced by organizations
• Consensus papers
• AMA
• ACEP
• NAEMTs

72. Evidence vs. Expert Opinion?
• If CONSENSU Sis the basis for a guideline, how is this distinguishable
from EVIDENCE-based guidelines?
• What are the implications of having these guidelines LOOK equally
authoritative when they make it to the street, when they ar ereally
two different levels of evidence?

74. Rational Conjecture
• Lowest level of scientific validity.
• But, overall very important.
• “You don’t have to run a Chi-
Square test on common sense”

76. What is Psuedo Science?
• Psuedo Science is weakly based conjecture often passed off as based
on science, often intentionally, with ulterior motives.
• Herbal Supplements
• “Tactical” medical products
• Just make it black and wityh tons of Velcro and someone will buy it!
• Targeted marketing
• “Hero”/”Ricky Rescue” complex
• It will often be supported with flashy presentations and plenty of
research, but when reviewed closely the research is conducted with
bias and conflicts of interests, and the presentations are weakly
interpreted extrapolations….

77. EBM: How does it fit together?
• Rare to find a body of knowledge that “writes the guideline for you.”
• Requires explicit cognitive steps that translate DIRECT evidence into
DIRECT guidelines through INFERENCES.
• From “Bench Test” to “Bedside Test”
• Example: Animal studies -> Human studies in limited settings -> Guideline applied
across a broad population in dramatically different settings.
• Inevitably requires judgment, extrapolation, experience, inference, and
opinion

78. Summary
• EMS must be driven by science.
• Science is based upon quality research.
• EMS providers of the future must be able to understand, and in
certain cases, conduct valid research.

79. Summary
• Evidence-Based Medicine:
• EMS must start adhering to the tenets of evidence-based medicine.
• Third-party payers will soon stop paying for care and procedures not
supported by science.
• Litigation may follow if non-evidence-based practices continue.

80. Evidence-Based Medicine
• Evidence-based medicine is not restricted to randomized controlled
trials and similar studies. It involves tracking down the best external
evidence with which to answer our clinical questions.

81. The End
• Don’t be afraid of research. It can
actually be fun!