This document discusses observational studies versus experimental design. It defines key terms like observational study, experiment, confounding variable, and provides examples. Randomized experiments are described as the gold standard for determining causation. Key aspects of experimental design like control groups, randomization, replication, and placebos are covered. Blocking and double-blind studies are introduced as ways to reduce bias.
2. Definitions
Observational Study – researchers simply observe or
question the participants about opinions, behaviors,
or outcomes. No treatment is imposed
Experiment – researchers manipulate something
and measure the effect of the manipulation on some
outcome of interest.
Confounding variable or lurking variable- a variable
that both affects the response variable and is also
related to the explanatory variable. Occurs more
often in observational studies
3. Examples of Observational Studies
Suppose that an observational study finds that
people who take at least 500 mg of vitamin C every
day get fewer colds than other people do.
Another observational study found that attending
church services extends the life span about as much
as moderate exercise or not smoking.
Another study found that a greater percentage of
Southerners have high blood pressure than do
people in any other region of the United States.
5. Randomized Experiments
Experimental Units – animals, plants, things.
People are called subjects or participants
Factors – explanatory variables
Level – combining specific value of each of the
factors (ex. Higher dosages of the same drug)
6. Example 1
Researchers studying the absorption of a drug into
the bloodstream inject the drug (the treatment) into
25 people. The response variable is the
concentration of the drug in a subject’s blood,
measured 30 minutes after the injection.
This experiment has a single factor with one level.
If three different doses of the drug are injected, there
is still a single factor (dosage of the drug), now with
three levels.
7. Example 2
A chemical engineer is designing 1. What are the explanatory and
the production process for a new response variables.
product. The chemical reaction
that produces the product may
have higher or lower yield,
depending on the temperature 2. How many factors are there?
and the stirring rate in the vessel
in which the reaction takes place.
The engineer decides to 3. List the treatments.
investigate the effects of
combinations of two
temperatures (50◦ C and 60◦ C)
and three stirring rates (60 rpm, 4. How many experimental units
90 rpm, and 120 rpm) on the are required for the experiment?
yield of the process. She will
process two batches of the
product at each combination of
temperature and stirring rate.
8. Placebo Effect
A placebo is a dummy treatment that can have no
physical effect. A response without actual treatment
is called the placebo effect.
Placebo Pills, Placebo Surgeries
9. Three principles of a controlled experiment
Control – control the effects of lurking variables such
as the placebo effect.
Randomization – randomly place participants in
groups, all experimental units are allocated at
random among all treatments.
Replication – repeat treatment on several subjects
(30 participants means treatment is repeated 30
times)
10. Example 3
A food company assesses 1. What is the factor(s)
the nutritional quality of a
new “instant breakfast”
product by feeding it to a
newly weaned male white
rats. The response variable 2. There are 30 rats
is a rat’s weight gain over a available for this
28 day period. A control experiment. Describe how
group of rats eats a to randomly decide to
standard diet but otherwise which treatment group
receives exactly the same they belong.
treatment as the
experimental group.
11. Statistically Significant
An observed effect is statistically significant when
the effect is too large to attribute plausibly to chance
variation.
12. Cautions
Hidden bias – remember that bias is systematically
favoring a certain outcome.
Lack of realism – experiment in a lab setting may not
be the same when implemented in the real world.
13. Double Blind Experiment
Neither the participant nor the person measuring or
evaluating the response is aware who receives the
treatment/placebo.
14. Block Experiment
A group of volunteers are sorted by some
characteristic before being placed randomly into
treatment groups.
Blocking helps to reduce the chances of that
characteristic from becoming a lurking variable
If we block by gender, then we suspect that men and
women may respond to treatment differently,
therefore we split them up separately to begin with.
Randomizing within blocks further reduces the
effects of lurking variables.
15. Blocking continued
When we block we are creating groups that are
similar.
This reduces variation…meaning the standard
deviation of the measurements will be smaller.
It will be easier for us to tell if our results are
significant because of the reduced variation.
16. Matched Pairs Design
1 – Two units are closely matched. A coin is flipped
to see which unit receives the treatment and which
one receives the placebo or standard treatment.
2 – One subject receives both treatments. A coin is
flipped to determine which treatment is tried first.
