1. Testing Multilevel Theories Through
Multilevel Structural Equation Modeling
Christopher R. Beasley
Midwest Eco 2013

2. Digital Copy of Slides
crbeasley.info

3. Structural Equation Modeling
Social
Desirability
Value
Congruence
Demands-
Abilities
Interpersonal
Similarity
Needs-
Supplies
Satisfaction
Commitment
OH TenureP-E Fit

4. Nesting
• Dependence
– Time
– Complex Sampling

5. Study Design
• Random sample by house
• Oxford House residents
– 95% of houses agreed to assist
• nj = 82
– 48% individual response rate
• ni = 296

6. Why Multilevel
• Disaggregation of data (Byrne, 2012)
– Biased estimates and standard errors
• Aggregated data (Byrne, 2012)
– Lack of individual variance may exaggerate group
effects

7. MSEM
• Extension of Multi-Group SEM
– Covariance matrices at within and between level
instead of for different groups
• Assumptions
– General Linear Model assumptions
• Linearity
• Normality
• Homoscedasticity
• Independence

8. MSEM Alternatives
• Segregated Approach (Yuan & Bentler, 2007)
– More established
– Modification indices
• Partially Saturated Approach (Ryu & West, 2009)
– MLR
– Random coefficients
– 2-1-1, 2-1-2, 2-2-2, 1-1-2, 1-2-1, 1-1-1
– Better power for level 2
– Greater n at L1, so more influence on model fit
statistics (Hox, 2002)

9. Software Packages
• Mplus
• LISREL
• EQS
• GLLAMM

10. Process (Stapleton, 2013)
1. Consider single-level model
2. Baseline models for w/i & b/t levels while
saturating the other level
3. Theoretical w/i model with b/t saturated for
model fit
4. Theoretical b/t model with w/i saturated for
model fit
5. Combined w/i & b/t theoretical model for
parameters
6. Evaluate random coefficients

11. 1. Consider Single Model
• Consider single-level model
– Null model for descriptives
• Define ICC < 0.02 as within unless very large
cluster sizes, then possibly lower threshold
WITHIN ARE OHFITVC;

12. Between-Group Variance
ANALYSIS: TYPE IS TWOLEVEL RANDOM;
MODEL:
%WITHIN%
OHFITVC;
%BETWEEN%
OHFITVC;
Observed Variables
Null
ICC
Value Congruence 0.03
Commitment 0.11
Citizenship Behavior 0.04

13. 1.1 Multilevel Reliability
ANALYSIS: TYPE IS TWOLEVEL;
MODEL:
%WITHIN%
VCw@1;
OHGEF3R OHGEF7R OHGEF15 (WR1-
WR3);
VCw BY OHGEF3R* (WL1)
OHGEF7R (WL2)
OHGEF15 (WL3);
%BETWEEN%
VCb@1;
OHGEF3R OHGEF7R OHGEF15 (BR1-
BR3);
VCb BY OHGEF3R* (BL1)
OHGEF7R (BL2)
OHGEF15 (BL3);
MODEL CONSTRAINT: NEW(NUMW
DENOMW OMEGAW NUMB DENOMB
OMEGAB);
NUMW = (WL1+WL2+WL3)**2;
DENOMW = NUMW+(WR1+WR2+WR3);
OMEGAW = NUMW/DENOMW;
NUMB = (BL1+BL2+BL3)**2;
DENOMB = NUMB+(BR1+BR2+BR3);
OMEGAB = NUMB/DENOMB;
WR1 > 0; BR1 > 0;
WR2 > 0; BR2 > 0;
WR3 > 0; BR3 > 0;
Geldhof, Preacher, & Zyhur (in press)

14. 1.1 Measurement Reliability
Measures ωw ωb
Satisfaction 0.77 0.92
Demands-Abilities Fit 0.78 0.97
Needs-Supplies Fit 0.79 0.92
Interpersonal Similarity 0.79 0.98
Social Desirability 0.88 0.96
Value Congruence 0.90 0.93
Tenure 0.91 0.93
Commitment 0.91 0.99

15. 1.2 Descriptive Statistics
DEFINE: SDESIR=SDESIR/3;
Observed Variables n Min Max Mean SD SE
Social Desirability 292 0 13 6.67 3.23 0.19
Tenure 291 1 5 1.78 0.81 0.05
Commitment 292 2 7 5.26 1.10 0.06
Satisfaction 293 2 7 6.14 1.01 0.06
Interpersonal Similarity 291 1 5 3.42 0.96 0.06
Value Congruence 291 1 5 3.88 0.76 0.04
Demands-Abilities 293 1 5 3.96 0.80 0.05
Needs-Supplies Fit 293 1 5 4.01 0.70 0.04

16. 2. Baseline Models
Baseline w/i Model
MODEL:
%WITHIN%
OHFITVC with OHCit@0 OHCOM@0;
OHCit with OHCOM@0;
%BETWEEN%
OHFITVC with OHCit OHCOM;
OHCit with OHCOM;
Baseline b/t Model
MODEL:
%WITHIN%
OHFITVC with OHCit OHCOM;
OHCit with OHCOM;
%BETWEEN%
OHFITVC with OHCit@0 OHCOM@0;
OHCit with OHCOM@0;

17. Commitment
3. Theoretical w/i Model
Value
Congruence
Citizenship
Behavior
Commitment
Value
Congruence
Citizenship
Behavior

18. 3. Theoretical w/i Model
ANALYSIS: TYPE IS TWOLEVEL;
MODEL:
%WITHIN%
OHCOM on OHFITVC;
OHCOM on OHCit;
%BETWEEN%
OHCOM with OHFITVC OHCit;
OHFITVC with OHCIT;

19. 3. Theoretical w/i Model
Chi-Square Test of Model Fit
Value 38.019*
Degrees of Freedom 1
P-Value 0.0000
Scaling Correction Factor
RMSEA
Estimate 0.359
CFI/TLI
CFI 0.895
TLI 0.373
SRMR
Value for Within 0.169
Value for Between 0.010

20. Commitment
4. Theoretical b/t Model
Value
Congruence
Citizenship
Behavior
Commitment
Value
Congruence
Citizenship
Behavior

21. 4. Theoretical b/t Model
ANALYSIS: TYPE IS TWOLEVEL;
MODEL:
%WITHIN%
OHCOM with OHFITVC OHCit;
OHFITVC with OHCIT;
%BETWEEN%
OHCOM OHFITVC OHCit;

22. 4. Theoretical b/t Model
Chi-Square Test of Model Fit
Value 5.192*
Degrees of Freedom 3
P-Value 0.1583
Scaling Correction Factor
RMSEA
Estimate 0.050
CFI/TLI
CFI 0.994
TLI 0.988
SRMR
Value for Within 0.022
Value for Between 0.693

23. Commitment
3. Theoretical Combined Model
Value
Congruence
Citizenship
Behavior
Commitment
Value
Congruence
Citizenship
Behavior
Commitment
Value
Congruence
Citizenship
Behavior

24. 5. Combined Model
ANALYSIS: TYPE IS TWOLEVEL;
MODEL:
%WITHIN%
OHCOM on OHFITVC;
OHCOM on OHCit;
%BETWEEN%
OHCOM OHFITVC OHCit;

25. 5. Combined Model
Two-Tailed
Estimate S.E. Est./S.E. P-Value
Within Level
OHCOM ON
OHFITVC 0.711 0.079 9.025 0.000
OHCIT 0.338 0.055 6.168 0.000
Between Level
Means
OHCOM 5.250 0.076 68.744 0.000
OHCIT 5.870 0.058 101.094 0.000
OHFITVC 3.887 0.046 83.593 0.000

26. Commitment
6. Random Coefficients
Value
Congruence
Citizenship
Behavior
Commitment
Value
Congruence
Citizenship
Behavior
Commitment
Value
Congruence
Citizenship
Behavior
Random
Slope

27. 6. Random Coefficients
LISTWISE=ON; (or Monte Carlo integration)
ANALYSIS: TYPE IS TWOLEVEL;
MODEL:
%WITHIN%
OHCOM on OHFITVC;
s1 | OHCOM ON OHCit;
%BETWEEN%
OHCOM OHFITVC OHCit;

28. 6. Random Coefficients
LISTWISE=ON;
ANALYSIS: TYPE IS TWOLEVEL;
MODEL:
%WITHIN%
OHCOM on OHFITVC;
s1 | OHCOM ON OHCit;
%BETWEEN%
OHCOM OHFITVC OHCit;
s1 on OHFITVC;

29. 6. Random Coefficients
Loglikelihood
H0 Value -1074.838
H0 Scaling Correction Factor 0.9723
for MLR
Information Criteria
Akaike (AIC) 2175.675
Bayesian (BIC) 2223.249
Sample-Size Adjusted BIC 2182.024

30. 6. Random Coefficients
Two-Tailed
Estimate S.E. Est./S.E. P-Value
Between Level
S1 ON
OHFITVC 0.327 0.112 2.905 0.004
Intercepts
S1 -1.099 0.454 -2.421 0.015

31. Unresolved Issues
• Sample & Power Estimation
• Model Fit Approaches
• Model Fit Statistics
• MLR Performance in Multilevel Model
• 3+ Levels
• Balancing of Cluster Sizes

32. Unresolved Issues - Saturation
• “In the MODEL command, the following variable
is a y-variable (endogenous) on the BETWEEN
level and an x-variable (exogenous) on the
WITHIN level.
• This variable will be treated as a y-variable on
both levels: OHGEFIS”
• Any discrepancy is treated as a y-variable on
both levels

33. Other Models
• Longitudinal Multilevel Models
• Latent Models
• Multilevel EFA
• Multilevel CFA
• 3+ Levels

34. Other Topics
• Convergence Problems
• Power Analysis & Sample Size
• Alternative Estimators (MLR default)
– MUML, Bayes
• Random Starting Seeds
• Interval Estimates
– Bayes
– Monte Carlo

35. Resources
• Barbara Byrne
Structural Equation Modeling with Mplus
• Hancock & Mueller
Structural Equation Modeling: A Second Course
• Little, Bovarid, & Card
Modeling Contextual Effects in Long. Studies
• Kris Preacher
http://www.quantpsy.org/pubs.htm
• Steve Miller “Things Statistical”
http://personalityandemotion.com/