Latent class (mixture) models are often used in wide range of fields. These models assume that the observed variables are independent given the latent classes: local independence. What if this assumption does not hold?

1. Example LCA Local dependence BVR and MI EPC Conclusions References
Detecting local dependence in
latent class models
Daniel Oberski
Department of methodology and statistics
(Based on joint work with Jeroen Vermunt and Geert Van Kollenburg)
Detecting local dependence in latent class models Daniel Oberski

2. Example LCA Local dependence BVR and MI EPC Conclusions References
Latent class analysis
• Latent class analysis (LCA) used for: model-based
classification, clustering, latent structure analysis,
estimating false positives and false negatives, (specificity
and sensitivity) of error-prone variables;
• assumes local independence;
Detecting local dependence in latent class models Daniel Oberski

3. Example LCA Local dependence BVR and MI EPC Conclusions References
Latent class analysis
• Latent class analysis (LCA) used for: model-based
classification, clustering, latent structure analysis,
estimating false positives and false negatives, (specificity
and sensitivity) of error-prone variables;
• assumes local independence;
• If local dependence, severe bias can occur.
Detecting local dependence in latent class models Daniel Oberski

4. Example LCA Local dependence BVR and MI EPC Conclusions References
Latent class analysis
• Latent class analysis (LCA) used for: model-based
classification, clustering, latent structure analysis,
estimating false positives and false negatives, (specificity
and sensitivity) of error-prone variables;
• assumes local independence;
• If local dependence, severe bias can occur.
Detecting local dependence in latent class models Daniel Oberski

5. Example LCA Local dependence BVR and MI EPC Conclusions References
Detecting local dependence in LCA
• Local dependence in LCA leads to model misfit;
• Usual reaction: increase number of classes;
This talk:
• Do not increase number of classes;
• Model the local dependencies directly;
Detecting local dependence in latent class models Daniel Oberski

6. Example LCA Local dependence BVR and MI EPC Conclusions References
Detecting local dependence in LCA
• Local dependence in LCA leads to model misfit;
• Usual reaction: increase number of classes;
This talk:
• Do not increase number of classes;
• Model the local dependencies directly;
• Use specifically tailored measures to detect which local
dependencies should be modeled:
Detecting local dependence in latent class models Daniel Oberski

7. Example LCA Local dependence BVR and MI EPC Conclusions References
Detecting local dependence in LCA
• Local dependence in LCA leads to model misfit;
• Usual reaction: increase number of classes;
This talk:
• Do not increase number of classes;
• Model the local dependencies directly;
• Use specifically tailored measures to detect which local
dependencies should be modeled:
• Bivariate residuals (BVR),
Detecting local dependence in latent class models Daniel Oberski

8. Example LCA Local dependence BVR and MI EPC Conclusions References
Detecting local dependence in LCA
• Local dependence in LCA leads to model misfit;
• Usual reaction: increase number of classes;
This talk:
• Do not increase number of classes;
• Model the local dependencies directly;
• Use specifically tailored measures to detect which local
dependencies should be modeled:
• Bivariate residuals (BVR),
• Score test or ``modification index'' (MI),
Detecting local dependence in latent class models Daniel Oberski

9. Example LCA Local dependence BVR and MI EPC Conclusions References
Detecting local dependence in LCA
• Local dependence in LCA leads to model misfit;
• Usual reaction: increase number of classes;
This talk:
• Do not increase number of classes;
• Model the local dependencies directly;
• Use specifically tailored measures to detect which local
dependencies should be modeled:
• Bivariate residuals (BVR),
• Score test or ``modification index'' (MI),
• Expected parameter change (EPC).
Detecting local dependence in latent class models Daniel Oberski

10. Example LCA Local dependence BVR and MI EPC Conclusions References
Detecting local dependence in LCA
• Local dependence in LCA leads to model misfit;
• Usual reaction: increase number of classes;
This talk:
• Do not increase number of classes;
• Model the local dependencies directly;
• Use specifically tailored measures to detect which local
dependencies should be modeled:
• Bivariate residuals (BVR),
• Score test or ``modification index'' (MI),
• Expected parameter change (EPC).
Detecting local dependence in latent class models Daniel Oberski

11. Example LCA Local dependence BVR and MI EPC Conclusions References
..1 Example LCA
..2 Local dependence
..3 BVR and MI
..4 EPC
..5 Conclusions
Detecting local dependence in latent class models Daniel Oberski

12. Example LCA Local dependence BVR and MI EPC Conclusions References
Example LCA: dentists rating caries in xrays
Detecting local dependence in latent class models Daniel Oberski

13. Example LCA Local dependence BVR and MI EPC Conclusions References
Example LCA: dentists rating caries in xrays
Detecting local dependence in latent class models Daniel Oberski

14. Example LCA Local dependence BVR and MI EPC Conclusions References
Example LCA: dentists rating caries in xrays
Here is an xray of a person who possibly has caries:
Detecting local dependence in latent class models Daniel Oberski

15. Example LCA Local dependence BVR and MI EPC Conclusions References
Example LCA: dentists rating caries in xrays
Detecting local dependence in latent class models Daniel Oberski

16. Example LCA Local dependence BVR and MI EPC Conclusions References
Example LCA: dentists rating caries in xrays
y1 : Yes (1)
y2 : No (0)
Detecting local dependence in latent class models Daniel Oberski

17. Example LCA Local dependence BVR and MI EPC Conclusions References
Example LCA: dentists rating caries in xrays
y1 : Yes (1)
y2 : No (0)
y3 : No (0)
Detecting local dependence in latent class models Daniel Oberski

18. Example LCA Local dependence BVR and MI EPC Conclusions References
Example LCA: dentists rating caries in xrays
y1 : Yes (1)
y2 : No (0)
y3 : No (0)
y4 : Yes (1)
Detecting local dependence in latent class models Daniel Oberski

19. Example LCA Local dependence BVR and MI EPC Conclusions References
Example LCA: dentists rating caries in xrays
y1 : Yes (1)
y2 : No (0)
y3 : No (0)
y4 : Yes (1)
Detecting local dependence in latent class models Daniel Oberski

20. Example LCA Local dependence BVR and MI EPC Conclusions References
Actual dentists' ratings collected by Espeland &
Handelman (1989)
> dentists
Var1 Var2 Var3 Var4 Var5 Observed
1 0 0 0 0 0 1880
2 0 0 0 0 1 789
3 0 0 0 1 0 43
4 0 0 0 1 1 75
5 0 0 1 0 0 23
6 0 0 1 0 1 63
7 0 0 1 1 0 8
8 0 0 1 1 1 22
9 0 1 0 0 0 188
10 0 1 0 0 1 191
11 0 1 0 1 0 17
... etc.
Detecting local dependence in latent class models Daniel Oberski

21. Example LCA Local dependence BVR and MI EPC Conclusions References
2-class local independence model for dentists
Detecting local dependence in latent class models Daniel Oberski

22. Example LCA Local dependence BVR and MI EPC Conclusions References
Local independence latent class model
Patternwise likelihood:
Pr(Y = y) =
∑
t
Pr(Y = y|ξ = t)Pr(ξ = t), (1)
where the conditional probability of the response patterns is
Pr(Y = y|ξ = t) =
exp(Xθ)
1T
exp(Xθ)
(2)
With X a design matrix containing:
• Observed variables main effects (intercepts);
• Latent class × observed variables interaction.
Detecting local dependence in latent class models Daniel Oberski

23. Example LCA Local dependence BVR and MI EPC Conclusions References
Estimation of local independence latent class model
Log-likelihood
ℓ(θ) = nT
log Pr(Y = y),
with vector n the observed frequency of each response
pattern. Total sample size is N. Maximum likelihood estimates
ˆθ = arg max
θ∈Rq
ℓ(θ)
by expectation-maximization, quasi-Newton, or a combination.
This gives expected frequencies ˆµ := N · Pr(Y = y|θ = ˆθ)
Detecting local dependence in latent class models Daniel Oberski

24. Example LCA Local dependence BVR and MI EPC Conclusions References
2-class local independence model for dentists' ratings
Possible goals:
Goal: How LCA is useful:
* See how well dentists rate xrays ``sensitivity'' & ``specificity''
(conditional probabilities)
from intercepts and slopes
* Classify each photograph Pattern posteriors give
probabilistic classification
* Estimate ``true'' caries prevalence Latent class proportions
Detecting local dependence in latent class models Daniel Oberski

25. Example LCA Local dependence BVR and MI EPC Conclusions References
2-class local independence model for dentists' ratings
Possible goals:
Goal: How LCA is useful:
* See how well dentists rate xrays ``sensitivity'' & ``specificity''
(conditional probabilities)
from intercepts and slopes
* Classify each photograph Pattern posteriors give
probabilistic classification
* Estimate ``true'' caries prevalence Latent class proportions
Local dependence biases all of these results.
Detecting local dependence in latent class models Daniel Oberski

26. Example LCA Local dependence BVR and MI EPC Conclusions References
Questions for you
• What does the latent class variable represent?
• And if the number of classes is increased?
Detecting local dependence in latent class models Daniel Oberski

27. Example LCA Local dependence BVR and MI EPC Conclusions References
Questions for you
• What does the latent class variable represent?
• And if the number of classes is increased?
Model fit:
L2 X2 df BIC AIC AIC3 CAIC
129.85 132.00 20 -35.37 89.85 69.85 -35.37
Detecting local dependence in latent class models Daniel Oberski

28. Example LCA Local dependence BVR and MI EPC Conclusions References
Questions for you
• What does the latent class variable represent?
• And if the number of classes is increased?
Model fit:
L2 X2 df BIC AIC AIC3 CAIC
129.85 132.00 20 -35.37 89.85 69.85 -35.37
• The model does not fit. What could be a reason?
• What should be done?
Detecting local dependence in latent class models Daniel Oberski

29. Example LCA Local dependence BVR and MI EPC Conclusions References
Local dependence latent class models
Detecting local dependence in latent class models Daniel Oberski

30. Example LCA Local dependence BVR and MI EPC Conclusions References
2-class local dependence model for dentists
Detecting local dependence in latent class models Daniel Oberski

31. Example LCA Local dependence BVR and MI EPC Conclusions References
Local dependence latent class model
Conditional probability of the response patterns still
Pr(Y = y|ξ = t) =
exp(Xθ)
1T
exp(Xθ)
But now X is a design matrix containing:
• Observed variables main effects (intercepts);
• Latent class × observed variables interaction;
• (Some) observed variables × observed variables 2-way
interactions;
Detecting local dependence in latent class models Daniel Oberski

32. Example LCA Local dependence BVR and MI EPC Conclusions References
• Can we find a locally dependent two-class model that fits?
• Which local dependence parameters are needed?
Detecting local dependence in latent class models Daniel Oberski

33. Example LCA Local dependence BVR and MI EPC Conclusions References
• Can we find a locally dependent two-class model that fits?
• Which local dependence parameters are needed?
Detecting local dependence in latent class models Daniel Oberski

34. Example LCA Local dependence BVR and MI EPC Conclusions References
• Can we find a locally dependent two-class model that fits?
• Which local dependence parameters are needed?
•• Goal: Examine fit of each pair of variables (dentist ratings)
to the local independence assumption without fitting all
possible alternative models.
Detecting local dependence in latent class models Daniel Oberski

35. Example LCA Local dependence BVR and MI EPC Conclusions References
• Can we find a locally dependent two-class model that fits?
• Which local dependence parameters are needed?
•• Goal: Examine fit of each pair of variables (dentist ratings)
to the local independence assumption without fitting all
possible alternative models.
Detecting local dependence in latent class models Daniel Oberski

36. Example LCA Local dependence BVR and MI EPC Conclusions References
Detecting local dependencies with the BVR and MI
Detecting local dependence in latent class models Daniel Oberski

37. Example LCA Local dependence BVR and MI EPC Conclusions References
Fit of two-way cross-table between dentists 1 and 3
Observed
No Yes
No 3250 280
Yes 123 216
Expected
No Yes
No 3217 313
Yes 156 183
Bivariate residuals
No Yes
No 32.6 -32.6
Yes -32.6 32.6
Detecting local dependence in latent class models Daniel Oberski

38. Example LCA Local dependence BVR and MI EPC Conclusions References
Fit of two-way cross-table between dentists 1 and 3
Observed
No Yes
No 3250 280
Yes 123 216
Expected
No Yes
No 3217 313
Yes 156 183
Bivariate residuals
No Yes
No 32.6 -32.6
Yes -32.6 32.6
BVR1,3 = r11
2
∑
k,l
ˆµ−1
kl
= (32.6)2
∑
k,l
ˆµ−1
kl
≈ 1063(0.0154) ≈ 16.3
Detecting local dependence in latent class models Daniel Oberski

39. Example LCA Local dependence BVR and MI EPC Conclusions References
Fit of two-way cross-table between dentists 1 and 3
Observed
No Yes
No 3250 280
Yes 123 216
Expected
No Yes
No 3217 313
Yes 156 183
Bivariate residuals
No Yes
No 32.6 -32.6
Yes -32.6 32.6
BVR1,3 = r11
2
∑
k,l
ˆµ−1
kl
= (32.6)2
∑
k,l
ˆµ−1
kl
≈ 1063(0.0154) ≈ 16.3
MI1,3 = r11
2
Var(r11)−1
= (32.6)2
/(31.3) ≈ 1063(0.0320) ≈ 34.0
Detecting local dependence in latent class models Daniel Oberski

40. Example LCA Local dependence BVR and MI EPC Conclusions References
Local dependencies between five dentists' x-ray
ratings for caries.
Dentist dependence MI BVR
1 ↔ 2 3.1 1.4
1 ↔ 3 34.0 ** 16.3
1 ↔ 4 13.1 ** 7.7
1 ↔ 5 2.7 0.8
2 ↔ 3 6.8 * 1.7
2 ↔ 4 1.8 0.6
2 ↔ 5 16.4 ** 4.7
3 ↔ 4 2.7 1.0
3 ↔ 5 5.1 * 0.8
4 ↔ 5 3.5 1.0
Note: ** Sig. (α = 0.05) with Bonferroni correction for multiple
testing, * without correction
Detecting local dependence in latent class models Daniel Oberski

41. Example LCA Local dependence BVR and MI EPC Conclusions References
BVR and MI
``Bivariate residual'' (BVR):
• Pearson "chi-square" residual in cross-table of a pair of
observed variables:
∑
(observed - expected)2/expected
• Surprise! Not a chi square statistic!
Detecting local dependence in latent class models Daniel Oberski

42. Example LCA Local dependence BVR and MI EPC Conclusions References
BVR and MI
``Bivariate residual'' (BVR):
• Pearson "chi-square" residual in cross-table of a pair of
observed variables:
∑
(observed - expected)2/expected
• Surprise! Not a chi square statistic!
Detecting local dependence in latent class models Daniel Oberski

43. Example LCA Local dependence BVR and MI EPC Conclusions References
q
q
q
q
q
q
q qq
q qq
qqqqqqqqqqqqqqqqqq
q
q q
q qq
q
q
q
q
q
q
0
20
40
60
80
100
0
20
40
60
80
100
MI equals chi−square improvement...
χ2
MI
q
q
q
q
q
q
q qq
q qq
qqqqqqqqqqqqqqqqqq q q q
q qq
q q qq
q
q
0
20
40
60
80
100
0
20
40
60
80
100
... BVR does not.
χ2
BVR
Detecting local dependence in latent class models Daniel Oberski

44. Example LCA Local dependence BVR and MI EPC Conclusions References
BVR and MI
``Bivariate residual'' (BVR):
• Pearson residual in cross-table of a pair of observed
variables: (observed - expected)2/expected;
• Not a chi square statistic.
``Modification index'' (MI):
• ``Score test'' (``Lagrange multiplier'' test) for introducing a
local dependency between a pair of variables;
• Turns out to use the same residual as the BVR, but with
the correct variance! (thanks Jeroen)
Detecting local dependence in latent class models Daniel Oberski

45. Example LCA Local dependence BVR and MI EPC Conclusions References
BVR and MI
``Bivariate residual'' (BVR):
• Pearson residual in cross-table of a pair of observed
variables: (observed - expected)2/expected;
• Not a chi square statistic.
``Modification index'' (MI):
• ``Score test'' (``Lagrange multiplier'' test) for introducing a
local dependency between a pair of variables;
• Turns out to use the same residual as the BVR, but with
the correct variance! (thanks Jeroen)
• Under the null hypothesis, chi-square distributed (1 df)
Detecting local dependence in latent class models Daniel Oberski

46. Example LCA Local dependence BVR and MI EPC Conclusions References
BVR and MI
``Bivariate residual'' (BVR):
• Pearson residual in cross-table of a pair of observed
variables: (observed - expected)2/expected;
• Not a chi square statistic.
``Modification index'' (MI):
• ``Score test'' (``Lagrange multiplier'' test) for introducing a
local dependency between a pair of variables;
• Turns out to use the same residual as the BVR, but with
the correct variance! (thanks Jeroen)
• Under the null hypothesis, chi-square distributed (1 df)
Detecting local dependence in latent class models Daniel Oberski

47. Example LCA Local dependence BVR and MI EPC Conclusions References
BVR not even close to chi-square distributed, MI is.
Monte Carlo simulation under the null hypothesis
Condition α for nominal 5% Empirical distribution
BVR MI BVR
λ n Naive Boot MI Mean Var Mean Var
0.5 200 0.000 0.050 0.051 0.97 1.7 0.33 0.2
0.5 500 0.000 0.020 0.050 1.06 2.3 0.36 0.2
0.5 1000 0.000 0.060 0.065 0.96 1.9 0.33 0.2
0.5 5000 0.000 0.085 0.055 0.97 2.0 0.34 0.2
0.8 200 0.000 0.065 0.040 1.04 1.7 0.25 0.1
0.8 500 0.000 0.070 0.060 1.05 2.0 0.25 0.1
0.8 1000 0.000 0.060 0.090 1.22 2.6 0.30 0.2
0.8 5000 0.000 0.035 0.060 1.16 3.1 0.28 0.2
Should be: 0.050 0.050 0.050 1.00 2.0 1.00 2.0
Detecting local dependence in latent class models Daniel Oberski

48. Example LCA Local dependence BVR and MI EPC Conclusions References
BVR and MI: main findings
• Many (most?) published applied studies using BVR
pretend it is a chi-square statistic*;
• BUT, if pretend BVR is chi-square:
* See paper.
Detecting local dependence in latent class models Daniel Oberski

49. Example LCA Local dependence BVR and MI EPC Conclusions References
BVR and MI: main findings
• Many (most?) published applied studies using BVR
pretend it is a chi-square statistic*;
• BUT, if pretend BVR is chi-square:
• BVR does not provide nominal α levels;
* See paper.
Detecting local dependence in latent class models Daniel Oberski

50. Example LCA Local dependence BVR and MI EPC Conclusions References
BVR and MI: main findings
• Many (most?) published applied studies using BVR
pretend it is a chi-square statistic*;
• BUT, if pretend BVR is chi-square:
• BVR does not provide nominal α levels;
• BVR power to detect local dependencies miserably low*.
* See paper.
Detecting local dependence in latent class models Daniel Oberski

51. Example LCA Local dependence BVR and MI EPC Conclusions References
BVR and MI: main findings
• Many (most?) published applied studies using BVR
pretend it is a chi-square statistic*;
• BUT, if pretend BVR is chi-square:
• BVR does not provide nominal α levels;
• BVR power to detect local dependencies miserably low*.
• Referring MI to chi-square:
* See paper.
Detecting local dependence in latent class models Daniel Oberski

52. Example LCA Local dependence BVR and MI EPC Conclusions References
BVR and MI: main findings
• Many (most?) published applied studies using BVR
pretend it is a chi-square statistic*;
• BUT, if pretend BVR is chi-square:
• BVR does not provide nominal α levels;
• BVR power to detect local dependencies miserably low*.
• Referring MI to chi-square:
• MI provides approximately nominal α levels;
* See paper.
Detecting local dependence in latent class models Daniel Oberski

53. Example LCA Local dependence BVR and MI EPC Conclusions References
BVR and MI: main findings
• Many (most?) published applied studies using BVR
pretend it is a chi-square statistic*;
• BUT, if pretend BVR is chi-square:
• BVR does not provide nominal α levels;
• BVR power to detect local dependencies miserably low*.
• Referring MI to chi-square:
• MI provides approximately nominal α levels;
• Power of MI is adequate except for very small
dependencies (±0.05)*.
* See paper.
Detecting local dependence in latent class models Daniel Oberski

54. Example LCA Local dependence BVR and MI EPC Conclusions References
BVR and MI: main findings
• Many (most?) published applied studies using BVR
pretend it is a chi-square statistic*;
• BUT, if pretend BVR is chi-square:
• BVR does not provide nominal α levels;
• BVR power to detect local dependencies miserably low*.
• Referring MI to chi-square:
• MI provides approximately nominal α levels;
• Power of MI is adequate except for very small
dependencies (±0.05)*.
* See paper.
Detecting local dependence in latent class models Daniel Oberski

55. Example LCA Local dependence BVR and MI EPC Conclusions References
BVR and MI: main findings
• Many (most?) published applied studies using BVR
pretend it is a chi-square statistic*;
• BUT, if pretend BVR is chi-square:
• BVR does not provide nominal α levels;
• BVR power to detect local dependencies miserably low*.
• Referring MI to chi-square:
• MI provides approximately nominal α levels;
• Power of MI is adequate except for very small
dependencies (±0.05)*.
•• Can bootstrap BVR values, leading to performance similar
to, though slightly below, MI*
* See paper.
Detecting local dependence in latent class models Daniel Oberski

56. Example LCA Local dependence BVR and MI EPC Conclusions References
BVR and MI: main findings
• Many (most?) published applied studies using BVR
pretend it is a chi-square statistic*;
• BUT, if pretend BVR is chi-square:
• BVR does not provide nominal α levels;
• BVR power to detect local dependencies miserably low*.
• Referring MI to chi-square:
• MI provides approximately nominal α levels;
• Power of MI is adequate except for very small
dependencies (±0.05)*.
•• Can bootstrap BVR values, leading to performance similar
to, though slightly below, MI*
* See paper.
Detecting local dependence in latent class models Daniel Oberski

57. Example LCA Local dependence BVR and MI EPC Conclusions References
Assessing substantive size of local dependencies with the EPC
Detecting local dependence in latent class models Daniel Oberski

58. Example LCA Local dependence BVR and MI EPC Conclusions References
Assessing substantive size besides statistical
siginificance
``Modification index'' (MI):
• Statistical ``score'' test of the hypothesis that the local
dependence between two variables, say ψ, equals zero.
• ψ corresponds to the observed variable × observed
variable 2-way interaction in conditional prob. model;
Detecting local dependence in latent class models Daniel Oberski

59. Example LCA Local dependence BVR and MI EPC Conclusions References
Assessing substantive size besides statistical
siginificance
``Modification index'' (MI):
• Statistical ``score'' test of the hypothesis that the local
dependence between two variables, say ψ, equals zero.
• ψ corresponds to the observed variable × observed
variable 2-way interaction in conditional prob. model;
• Only statistical significance → MI depends on:
Detecting local dependence in latent class models Daniel Oberski

60. Example LCA Local dependence BVR and MI EPC Conclusions References
Assessing substantive size besides statistical
siginificance
``Modification index'' (MI):
• Statistical ``score'' test of the hypothesis that the local
dependence between two variables, say ψ, equals zero.
• ψ corresponds to the observed variable × observed
variable 2-way interaction in conditional prob. model;
• Only statistical significance → MI depends on:
• Sample size;
Detecting local dependence in latent class models Daniel Oberski

61. Example LCA Local dependence BVR and MI EPC Conclusions References
Assessing substantive size besides statistical
siginificance
``Modification index'' (MI):
• Statistical ``score'' test of the hypothesis that the local
dependence between two variables, say ψ, equals zero.
• ψ corresponds to the observed variable × observed
variable 2-way interaction in conditional prob. model;
• Only statistical significance → MI depends on:
• Sample size;
• Size of the loadings of the two variables;
Detecting local dependence in latent class models Daniel Oberski

62. Example LCA Local dependence BVR and MI EPC Conclusions References
Assessing substantive size besides statistical
siginificance
``Modification index'' (MI):
• Statistical ``score'' test of the hypothesis that the local
dependence between two variables, say ψ, equals zero.
• ψ corresponds to the observed variable × observed
variable 2-way interaction in conditional prob. model;
• Only statistical significance → MI depends on:
• Sample size;
• Size of the loadings of the two variables;
• Size of the loadings of the other variables;
Detecting local dependence in latent class models Daniel Oberski

63. Example LCA Local dependence BVR and MI EPC Conclusions References
Assessing substantive size besides statistical
siginificance
``Modification index'' (MI):
• Statistical ``score'' test of the hypothesis that the local
dependence between two variables, say ψ, equals zero.
• ψ corresponds to the observed variable × observed
variable 2-way interaction in conditional prob. model;
• Only statistical significance → MI depends on:
• Sample size;
• Size of the loadings of the two variables;
• Size of the loadings of the other variables;
• Latent variable intercept(s);
Detecting local dependence in latent class models Daniel Oberski

64. Example LCA Local dependence BVR and MI EPC Conclusions References
Assessing substantive size besides statistical
siginificance
``Modification index'' (MI):
• Statistical ``score'' test of the hypothesis that the local
dependence between two variables, say ψ, equals zero.
• ψ corresponds to the observed variable × observed
variable 2-way interaction in conditional prob. model;
• Only statistical significance → MI depends on:
• Sample size;
• Size of the loadings of the two variables;
• Size of the loadings of the other variables;
• Latent variable intercept(s);
• Observed variable intercepts;
Detecting local dependence in latent class models Daniel Oberski

65. Example LCA Local dependence BVR and MI EPC Conclusions References
Assessing substantive size besides statistical
siginificance
``Modification index'' (MI):
• Statistical ``score'' test of the hypothesis that the local
dependence between two variables, say ψ, equals zero.
• ψ corresponds to the observed variable × observed
variable 2-way interaction in conditional prob. model;
• Only statistical significance → MI depends on:
• Sample size;
• Size of the loadings of the two variables;
• Size of the loadings of the other variables;
• Latent variable intercept(s);
• Observed variable intercepts;
• Regression coefficients of covariates if present.
Detecting local dependence in latent class models Daniel Oberski

66. Example LCA Local dependence BVR and MI EPC Conclusions References
Assessing substantive size besides statistical
siginificance
``Modification index'' (MI):
• Statistical ``score'' test of the hypothesis that the local
dependence between two variables, say ψ, equals zero.
• ψ corresponds to the observed variable × observed
variable 2-way interaction in conditional prob. model;
• Only statistical significance → MI depends on:
• Sample size;
• Size of the loadings of the two variables;
• Size of the loadings of the other variables;
• Latent variable intercept(s);
• Observed variable intercepts;
• Regression coefficients of covariates if present.
Detecting local dependence in latent class models Daniel Oberski

67. Example LCA Local dependence BVR and MI EPC Conclusions References
Assessing substantive size besides statistical
siginificance
``Expected parameter change'' (EPC):
• Estimate of ψ, the strength of the local dependence
between two variables;
• No need to fit alternative model;
Detecting local dependence in latent class models Daniel Oberski

68. Example LCA Local dependence BVR and MI EPC Conclusions References
Assessing substantive size besides statistical
siginificance
``Expected parameter change'' (EPC):
• Estimate of ψ, the strength of the local dependence
between two variables;
• No need to fit alternative model;
• Assesses substantive rather than statistical significance.
Detecting local dependence in latent class models Daniel Oberski

69. Example LCA Local dependence BVR and MI EPC Conclusions References
Assessing substantive size besides statistical
siginificance
``Expected parameter change'' (EPC):
• Estimate of ψ, the strength of the local dependence
between two variables;
• No need to fit alternative model;
• Assesses substantive rather than statistical significance.
Detecting local dependence in latent class models Daniel Oberski

70. Example LCA Local dependence BVR and MI EPC Conclusions References
Local dependencies between five dentists' x-ray
ratings for caries.
Dentist dependence EPC MI BVR
1 ↔ 2 -0.081 3.1 1.4
1 ↔ 3 -0.261 34.0 ** 16.3
1 ↔ 4 -0.146 13.1 ** 7.7
1 ↔ 5 -0.117 2.7 0.8
2 ↔ 3 -0.140 6.8 * 1.7
2 ↔ 4 -0.058 1.8 0.6
2 ↔ 5 -0.157 16.4 ** 4.7
3 ↔ 4 0.074 2.7 1.0
3 ↔ 5 -0.191 5.1 * 0.8
4 ↔ 5 -0.104 3.5 1.0
Note: ** Sig. (α = 0.05) with Bonferroni correction for multiple
testing, * without correction
Detecting local dependence in latent class models Daniel Oberski

71. Example LCA Local dependence BVR and MI EPC Conclusions References
Based on EPC and MI,
• keep two-classes, but...
• ...free five out of ten possible bivariate local
dependencies.
Detecting local dependence in latent class models Daniel Oberski

72. Example LCA Local dependence BVR and MI EPC Conclusions References
Based on EPC and MI,
• keep two-classes, but...
• ...free five out of ten possible bivariate local
dependencies.
Detecting local dependence in latent class models Daniel Oberski

73. Example LCA Local dependence BVR and MI EPC Conclusions References
Based on EPC and MI,
• keep two-classes, but...
• ...free five out of ten possible bivariate local
dependencies.
•• 15 degrees of freedom (was 20)
Detecting local dependence in latent class models Daniel Oberski

74. Example LCA Local dependence BVR and MI EPC Conclusions References
Based on EPC and MI,
• keep two-classes, but...
• ...free five out of ten possible bivariate local
dependencies.
•• 15 degrees of freedom (was 20)
• L2
= 28.4 (p = 0.07) (was 129.9)
Detecting local dependence in latent class models Daniel Oberski

75. Example LCA Local dependence BVR and MI EPC Conclusions References
Based on EPC and MI,
• keep two-classes, but...
• ...free five out of ten possible bivariate local
dependencies.
•• 15 degrees of freedom (was 20)
• L2
= 28.4 (p = 0.07) (was 129.9)
• BIC is −95.5 (was −35.37)
Detecting local dependence in latent class models Daniel Oberski

76. Example LCA Local dependence BVR and MI EPC Conclusions References
Based on EPC and MI,
• keep two-classes, but...
• ...free five out of ten possible bivariate local
dependencies.
•• 15 degrees of freedom (was 20)
• L2
= 28.4 (p = 0.07) (was 129.9)
• BIC is −95.5 (was −35.37)
• Best two-class model in lit. (Qu et al. 1996): BIC −83.4
Detecting local dependence in latent class models Daniel Oberski

77. Example LCA Local dependence BVR and MI EPC Conclusions References
Based on EPC and MI,
• keep two-classes, but...
• ...free five out of ten possible bivariate local
dependencies.
•• 15 degrees of freedom (was 20)
• L2
= 28.4 (p = 0.07) (was 129.9)
• BIC is −95.5 (was −35.37)
• Best two-class model in lit. (Qu et al. 1996): BIC −83.4
Detecting local dependence in latent class models Daniel Oberski

78. Example LCA Local dependence BVR and MI EPC Conclusions References
Conclusions
Detecting local dependence in latent class models Daniel Oberski

79. Example LCA Local dependence BVR and MI EPC Conclusions References
Conclusions
• Local dependence is a serious problem in LCA;
• Sometimes modeling dependence directly is preferable to
increasing no. classes;
Detecting local dependence in latent class models Daniel Oberski

80. Example LCA Local dependence BVR and MI EPC Conclusions References
Conclusions
• Local dependence is a serious problem in LCA;
• Sometimes modeling dependence directly is preferable to
increasing no. classes;
• Need to monitor local dependence.
Detecting local dependence in latent class models Daniel Oberski

81. Example LCA Local dependence BVR and MI EPC Conclusions References
Conclusions
• Local dependence is a serious problem in LCA;
• Sometimes modeling dependence directly is preferable to
increasing no. classes;
• Need to monitor local dependence.
Detecting local dependence in latent class models Daniel Oberski

82. Example LCA Local dependence BVR and MI EPC Conclusions References
Conclusions
• Local dependence is a serious problem in LCA;
• Sometimes modeling dependence directly is preferable to
increasing no. classes;
• Need to monitor local dependence.
•• Introduced MI and BVR, tests for local dependence:
Detecting local dependence in latent class models Daniel Oberski

83. Example LCA Local dependence BVR and MI EPC Conclusions References
Conclusions
• Local dependence is a serious problem in LCA;
• Sometimes modeling dependence directly is preferable to
increasing no. classes;
• Need to monitor local dependence.
•• Introduced MI and BVR, tests for local dependence:
• MI works well as chi-square statistic;
Detecting local dependence in latent class models Daniel Oberski

84. Example LCA Local dependence BVR and MI EPC Conclusions References
Conclusions
• Local dependence is a serious problem in LCA;
• Sometimes modeling dependence directly is preferable to
increasing no. classes;
• Need to monitor local dependence.
•• Introduced MI and BVR, tests for local dependence:
• MI works well as chi-square statistic;
• BVR does not, but can bootstrap p-values.
Detecting local dependence in latent class models Daniel Oberski

85. Example LCA Local dependence BVR and MI EPC Conclusions References
Conclusions
• Local dependence is a serious problem in LCA;
• Sometimes modeling dependence directly is preferable to
increasing no. classes;
• Need to monitor local dependence.
•• Introduced MI and BVR, tests for local dependence:
• MI works well as chi-square statistic;
• BVR does not, but can bootstrap p-values.
• Introduced EPC: local dependence substantive size.
Detecting local dependence in latent class models Daniel Oberski

86. Example LCA Local dependence BVR and MI EPC Conclusions References
Conclusions
• Local dependence is a serious problem in LCA;
• Sometimes modeling dependence directly is preferable to
increasing no. classes;
• Need to monitor local dependence.
•• Introduced MI and BVR, tests for local dependence:
• MI works well as chi-square statistic;
• BVR does not, but can bootstrap p-values.
• Introduced EPC: local dependence substantive size.
Detecting local dependence in latent class models Daniel Oberski

87. Example LCA Local dependence BVR and MI EPC Conclusions References
Conclusions
• Local dependence is a serious problem in LCA;
• Sometimes modeling dependence directly is preferable to
increasing no. classes;
• Need to monitor local dependence.
•• Introduced MI and BVR, tests for local dependence:
• MI works well as chi-square statistic;
• BVR does not, but can bootstrap p-values.
• Introduced EPC: local dependence substantive size.
•• Dentist example: obtained more parsimonious and
easy-to-interpret two-class model than current literature.
Detecting local dependence in latent class models Daniel Oberski

88. Example LCA Local dependence BVR and MI EPC Conclusions References
Conclusions
• Local dependence is a serious problem in LCA;
• Sometimes modeling dependence directly is preferable to
increasing no. classes;
• Need to monitor local dependence.
•• Introduced MI and BVR, tests for local dependence:
• MI works well as chi-square statistic;
• BVR does not, but can bootstrap p-values.
• Introduced EPC: local dependence substantive size.
•• Dentist example: obtained more parsimonious and
easy-to-interpret two-class model than current literature.
Detecting local dependence in latent class models Daniel Oberski

89. Example LCA Local dependence BVR and MI EPC Conclusions References
• Class-specific local dependence;
• Multivariate MI, EPC, (polytomous items);
• Effect of freeing local dependencies on params of interest;
• Problem of parameter dependence;
• Problem of multiple comparisons;
• (Post-hoc) power of MI (Satorra);
• MI for other params than locdeps (e.g. item bias).
Detecting local dependence in latent class models Daniel Oberski

90. Example LCA Local dependence BVR and MI EPC Conclusions References
Thank you for your attention!
Daniel Oberski
doberski@uvt.nl
Working papers on this topic:
Oberski, D., Van Kollenburg, G., and Vermunt, J. (submitted). A
Monte Carlo evaluation of three methods to detect local
dependence in binary data latent class models.
Oberski, D. and Vermunt, J. (submitted). The Expected Parameter
Change (EPC) for local dependence assessment in binary data
latent class models.
Oberski, D. (submitted). Change in SEM parameters of interest as
a criterion for partial measurement invariance: The
EPC-interest.
Detecting local dependence in latent class models Daniel Oberski

91. Appendix
Appendix
Detecting local dependence in latent class models Daniel Oberski

92. Appendix
0.5 0.8
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
−0.4−0.2−0.050.050.20.4
200
500
1000
5000
200
500
1000
5000
Log(sample size)
Power
p−value type
BVR
BVR_bootstrap
MI
Detecting local dependence in latent class models Daniel Oberski

93. Appendix
Convergence to noncentral chi-square in worst
condition
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qq
q
q
Sample size: 200
Q−Q plot, noncentral χ1
2
(4.7)
Theoretical quantiles
MI
0
10
20
30
40
0 5 10 15 20 25
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
q q
Sample size: 500
Q−Q plot, noncentral χ1
2
(11.8)
Theoretical quantiles
MI
0
10
20
30
40
50
60
70
0 10 20 30 40
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
Sample size: 1000
Q−Q plot, noncentral χ1
2
(23.7)
Theoretical quantiles
MI
0
20
40
60
80
10 20 30 40 50 60
q
qqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqqq
qqqqq
qq
q
Sample size: 5000
Q−Q plot, noncentral χ1
2
(118)
Theoretical quantiles
MI
50
100
150
200
80 120 160
Detecting local dependence in latent class models Daniel Oberski

94. Appendix
Local dependencies between indicators of Hispanic
ethnicity in the U.S. Census
Local dependence EPCL TL EPCgs TGS
Ancestry-re ↔ Language-in 0.92 5.0 1.45 7.9 0.
Ancestry-re ↔ Origin-in -0.76 2.5 -1.23 4.1 -0.
Ancestry-re ↔ Origin-re 2.94 45.6 1.32 20.5 1.
Language-in ↔ Origin-in 4.14 97.1 1.59 37.2 3.
Language-in ↔ Origin-re -1.08 7.9 -1.76 12.8 1.
Origin-in ↔ Origin-re 1.10 6.1 2.20 12.2 0.
Detecting local dependence in latent class models Daniel Oberski