Bayesian Model Averaging for Structural Equation Models

• Applicant: Dr. Julius M. Pfadt
• Subject Area: General, Cognitive and Mathematical Psychology
• Term: since 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 531600276

In psychology, researchers often use Structural Equation Modeling (SEM) to explore the relationships between unobservable psychological constructs modeling them as latent variables. When testing if a SEM model fits the data, researchers usually consider several competing models but end up choosing and reporting a single model on which they base all conclusions. This practice ignores the uncertainty in the model selection process and conceals that alternative competitive models exist. Researchers risk discarding plausible alternative models and may likely report models that fail to replicate. In applications, the reliance on a single model leads to making overconfident and potentially misleading judgments about the relations between psychological constructs To address this issue, I propose to apply Bayesian model averaging (BMA) to SEM. BMA addresses the uncertainty in model selection by including all candidate models in the analysis at all times and weighting each model's impact based on its probability given the data (posterior probability). Weighting each model's impact allows taking into account that not all candidate models are equally likely. Models that fit the data well are weighted with a higher posterior probability than models that do not. This way, even though all candidate models are retained, less likely models have a smaller impact on model inferences. For instance, inferring the relationship (path) between two latent variables in a SEM model may be done by averaging the path over all candidate models. Specifically, when computing the average the path estimate from each model is weighted by the posterior probability, giving models with higher probability more influence on the resulting average path. Thus, in contrast to current practice, BMA allows making inferences based on the full range of candidate models instead of a single model. By not discarding the candidate models before drawing inferences about model parameters, BMA carries over the uncertainty in the model selection process to the inference stage. This way, BMA for SEM avoids overconfident, hasty, and possibly misleading conclusions, and increases predictive performance. Currently, researchers applying SEM are unable to benefit from BMA because they lack a comprehensive methodology and software implementation. Therefore, the objective of this project is to create and implement statistical tools to perform BMA for a broad range of structural equation models unlocking the power of BMA for SEM applications. By providing these tools through open-source statistical software, that is, R and JASP, the project offers substantive researchers accessible techniques to account for the uncertainty in model selection. Thus, the project will improve the inferences drawn from SEM models, increase the predictive power of SEM, and boost replication efforts.

• DFG Programme: WBP Fellowship
• International Connection: Netherlands
• Host: Professor Dr. Eric-Jan Wagenmakers