1. Week 8: Type I Error and Type
II Error
Statistical Power
1

2. Type I and Type II error:
 When we conduct hypothesis testing, we either reject or fail to
reject the null hypothesis. Our decision usually causes four
outcomes:
1. Reject the null hypothesis when it is false.
2. Keep the null hypothesis when it is true.
3. Reject the null hypothesis when it is true. (Type I error: alpha
α)
4. Fail to reject the null hypothesis when it is false. (Type II error:
beta β )
2

3. Type I and Type II error:
State of Nature
Decision Null true Null false
Made
Reject Null Type I error Correct decision
(α) (1 – b) POWER!
Fail to reject Correct Type II error (b)
null decision
3

4. Another example:
TRUTH
Innocent Guilty
Reject Incorrect decision Correct decision (1 –
null: Find Type 1 error (α) b) or Power
Guilty!
Fail to Correct decision Incorrect decision
reject Type II error (b)
null: Find
Innocent
4

5. Scenario 1:
Null: Reality: Conclusion: Decision:
There is no Null is true: Fail to reject the Correct
difference There is no null: Conclude decision.
or difference or there is no
relationship. relationship. difference or
relationship.
We failed to reject the null hypothesis when it is in
fact true.
In reality, there is no significant difference or
relationship to find in this case.
5

6. Scenario 2:
Null: Reality: Conclusion: Decision:
There is no Null is false: Reject the null: Correct
difference There is a Conclude there is decision.
or difference or a difference or POWER! (1 –
relationship. relationship. relationship. b)
We reject the null hypothesis when it is in fact false.
In reality, there is a significant difference or
relationship to find in this case.
6

7. Scenario 3:
Null: Reality: Conclusion: Decision:
There is no Null is false: Fail to reject the Incorrect
difference There is a null: Conclude Decision: Type
or difference or that there is no II error (b)
relationship. relationship. difference or
relationship.
We failed to reject the null hypothesis when it is
false.
In reality, there was a significant difference or
relationship to find in this case and we didn’t find it.
7

8. Scenario 4:
Null: Reality: Conclusion: Decision:
There is no Null is true: Reject the null: Incorrect
difference or There is no Conclude there is Decision: Type
relationship. difference or a difference or I error (α)
relationship. relationship.
We reject the null hypothesis when it was in fact true.
In reality, there was not a significant difference or relationship
to find in this case and we found a difference or relationship.
8

9. Type I and Type II error:
 A Type I Error is the false rejection of
a true null. It has a probability equal to
alpha ( α ).
 A Type II Error is the false retention
of a false null. It has a probability equal
to beta ( β ).
9

10. Statistical power:
 Power is the probability of correctly
rejecting a false null hypothesis. It’s
represented by 1- β .
 Factors affecting statistical power:
3. Sample size
4. Effect size
5. Alpha level
6. Directionality
10

11. Sample Size
 Sample size affects power. All else being equal, a
larger sample size yields more power than a smaller
sample size.
11

12. Effect Size
 Power is influenced by effect size. Effect size is the difference
between the value of the null and the value of the alternative,
or the desired difference to be detected. All else being equal, a
test is more powerful with a larger effect size.
12

13. Alpha Level
 The level of significance, or alpha, influences power.
All else being equal, a higher alpha yields higher
power. Does this mean we necessarily want to set a
high alpha level?
13

14. Directionality
 Power is affected by directionality of the test. All else
being equal, a one-tailed test is more powerful than
a two-tailed test. Why?
14

15. In-Class Activity:
Group 1: Influence of alpha level on power
Group 2: Influence of sample size on power
Group 3: Influence of effect size on power
Group 4: Influence of directionality on power
15