Equivalence Testing to Judge Model Fit: A Monte Carlo Simulation

Document Type

Article

Journal/Book Title/Conference

Psychological Methods

Publisher

American Psychological Association

Publication Date

1-1-2023

First Page

1

Last Page

38

Abstract

Decades of published methodological research have shown the chi-square test of model fit performs inconsistently and unreliably as a determinant of structural equation model (SEM) fit. Likewise, SEM indices of model fit, such as comparative fit index (CFI) and root-mean-square error of approximation (RMSEA) also perform inconsistently and unreliably. Despite rather unreliable ways to statistically assess model fit, researchers commonly rely on these methods for lack of a suitable inferential alternative. Marcoulides and Yuan (2017) have proposed the first inferential test of SEM fit in many years: an equivalence test adaptation of the RMSEA and CFI indicies (i.e., RMSEA t and CFI t). However, the ability of this equivalence testing approach to accurately judge acceptable and unacceptable model fit has not been empirically tested. This fully crossed Monte Carlo simulation evaluated the accuracy of equivalence testing combining many of the same independent variable (IV) conditions used in previous fit index simulation studies, including sample size (N = 100-1,000), model specification (correctly specified or misspecified), model type (confirmatory factor analysis [CFA], path analysis, or SEM), number of variables analyzed (low or high), data distribution (normal or skewed), and missing data (none, 10%, or 25%). Results show equivalence testing performs rather inconsistently and unreliably across IV conditions, with acceptable or unacceptable RMSEA t and CFIt model fit index values often being contingent on complex interactions among conditions. Proportional z-tests and logistic regression analyses indicated that equivalence tests of model fit are problematic under multiple conditions, especially those where models are mildly misspecified. Recommendations for researchers are offered, but with the provision that they be used with caution until more research and development is available.

Link to Full Text

Share

COinS