Project Details
Projekt
A Bayesian model framework for analyzing data from longitudinal large-scale assessments
Applicants
Professor Dr. Christian Aßmann, Ph.D.; Professor Dr. Claus Carstensen; Professorin Dr. Steffi Pohl
Subject Area
Empirical Social Research
Statistics and Econometrics
Statistics and Econometrics
Term
from 2014 to 2021
Project identifier
Deutsche Forschungsgemeinschaft (DFG) - Project number 263946158
Drawing inferences from educational panel studies poses several challenges. The aim of our project has been to address these challenges and to develop a model framework for the analyses of data from longitudinal large-scale assessments (LSAs), such as the National Educational Panel Study (NEPS). We developed a Bayesian estimation approach for Item Response Theory and Confirmatory Item Factor models with binary or ordinal responses, in which competencies are characterized as latent variables while simultaneously accounting for missing values in background variables. In order to account for the longitudinal structure, we developed strategies for the identification of anchor items needed for linking and evaluated them in simulation studies and empirical data analyses. We investigated different calibration approaches for linking longitudinal Rasch scaled competence test data and incorporated models for linking and longitudinal analyses in the overall analyses framework. Our approach also incorporates hierarchical structures in form of random effects, multiple groups, or latent classes. The hierarchical models of our framework have been applied to NEPS data for investigating effects of the stratified structure of the German school system on competence measures and test administrator effects on mathematics achievement.This research proposal describes further enhancements to the developed framework for the upcoming two-year funding period. First, we will implement statistical approaches arising in the Bayesian context for the comparison and averaging of non-nested model specifications. Specifically, we will implement the computation of the marginal likelihood given as the normalizing of the posterior distribution and building on these Bayesian model averaging. We will make use of Bayesian model averaging in the development of an anchor item approach for linking purposes. Second, the model framework is extended to deal with log data. Specifically, we will incorporate recent innovations in response time modeling allowing for a more efficient competence estimation, as well as for modeling test taking behavior. Third, we will extend our framework with regard to selection of background variables in a longitudinal setting for the estimation of plausible values. As approaches developed for cross sectional methods are inefficient when adapted for longitudinal designs with a huge amount of variables cumulated over survey waves, we plan to incorporate automated variable selection procedures. By further enhancing our current framework to incorporate model comparison and averaging, new linking approaches, the inclusion of response time models, and automated variable selection for estimating plausible values, our framework is applicable to a wider range of research questions typically addressed in LSAs. Our modeling framework will be available to NEPS data users and we will demonstrate its use in applications using NEPS data.
DFG Programme
Infrastructure Priority Programmes
