Lecture slides stats1.13.l23.air

1. Statistics One
Lecture 23
Generalized Linear Model

2. Two segments
• Overview
• Examples
2

3. Lecture 23 ~ Segment 1
Generalized Linear Model
Overview

4. Generalized Linear Model
• An extension of the General Linear Model
that allows for non-normal distributions in
the outcome variable and therefore also
allows testing of non-linear relationships
between a set of predictors and the outcome
variable
4

5. Generalized Linear Model
• Generalized Linear Model: GLM*
• General Linear Model: GLM
5

6. General Linear Model (GLM)
• GLM is the mathematical framework used in
many common statistical analyses, including
multiple regression and ANOVA
6

7. Characteristics of GLM
• Linear: pairs of variables are assumed to
have linear relations
• Additive: if one set of variables predict
another variable, the effects are thought to
be additive
7

8. Characteristics of GLM
• BUT! This does not preclude testing nonlinear or non-additive effects
8

9. Characteristics of GLM
• GLM can accommodate such tests, for
example, by
• Transformation of variables
– Transform so non-linear becomes linear
• Moderation analysis
– Fake the GLM into testing non-additive effects
9

10. GLM example
• Simple regression
• Y = B0 + B1X1 + e
• Y = faculty salary
• X1 = years since PhD
10

11. GLM example
• Multiple regression
• Y = B0 + B1X1 + B2X2 + B3X3 + e
• Y = faculty salary
• X1 = years since PhD
• X2 = number of publications
• X3 = (years x pubs)
11

12. Generalized linear model (GLM*)
• Appropriate when simple transformations or
product terms are not sufficient
12

13. Generalized linear model (GLM*)
• The “linear” model is allowed to generalize
to other forms by adding a “link function”
13

14. Generalized linear model (GLM*)
• For example, in binary logistic regression,
the logit function was the link function
14

15. Binary logistic regression
• ln(Ŷ / (1 - Ŷ)) = B0 + Σ(BkXk)
Ŷ = predicted value on the outcome variable Y
B0 = predicted value on Y when all X = 0
Xk = predictor variables
Bk = unstandardized regression coefficients
(Y – Ŷ) = residual (prediction error)
k = the number of predictor variables
15

16. Segment summary
• GLM* is an extension of GLM that allows for
non-normal distributions in the outcome
variable and therefore also allows testing of
non-linear relationships between a set of
predictors and the outcome variable
16

17. Segment summary
• Appropriate when simple transformations or
product terms are not sufficient
17

18. Segment summary
• The “linear” model is allowed to generalize
to other forms by adding a “link function”
18

19. END SEGMENT

20. Lecture 23 ~ Segment 2
Generalized Linear Model
Examples

21. GLM* Examples
• GLM* is an extension of GLM that allows for
non-normal distributions in the outcome
variable and therefore also allows testing of
non-linear relationships between a set of
predictors and the outcome variable
21

22. GLM* Examples
• Appropriate when simple transformations or
product terms are not sufficient
22

23. GLM* Examples
• The “linear” model is allowed to generalize
to other forms by adding a “link function”
23

24. GLM* Examples
• Binary logistic regression
24

25. Binary logistic regression

26. GLM* Examples
• In binary logistic regression, the logit
function served as the link function
26

27. GLM* Examples
• ln(Ŷ / (1 - Ŷ)) = B0 + Σ(BkXk)
Ŷ = predicted value on the outcome variable Y
B0 = predicted value on Y when all X = 0
Xk = predictor variables
Bk = unstandardized regression coefficients
(Y – Ŷ) = residual (prediction error)
k = the number of predictor variables
27

28. GLM* Examples
• More than 2 categories on the outcome
– Multinomial logistic regression
• A-1 logistic regression equations are formed
– Where A = # of groups
– One group serves as reference group

29. GLM* Examples
• Another example is Poisson regression
• Poission distributions are common with
“count data”
– A number of events occurring in a fixed interval
of time
29

30. GLM* Examples
• For example, the number of traffic accidents
as a function of weather conditions
– Clear weather
– Rain
– Snow
30

31. Poisson regression
31

32. GLM* Examples
• In Poisson regression, the log function
serves as the link function
• Note: this example also has a categorical
predictor and would therefore also require
dummy coding
32

33. Segment summary
• GLM* is an extension of GLM that allows for
non-normal distributions in the outcome
variable and therefore also allows testing of
non-linear relationships between a set of
predictors and the outcome variable
33

34. Segment summary
• Appropriate when simple transformations or
product terms are not sufficient
34

35. Segment summary
• The “linear” model is allowed to generalize
to other forms by adding a “link function”
35

36. END SEGMENT

37. END LECTURE 23