Optimized estimation of multilevel models in research on learning and instruction

• Applicants: Professor Dr. Martin Hecht; Professor Dr. Steffen Zitzmann
• Subject Area: Developmental and Educational Psychology
• Term: since 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 471861494

Researchers in the field of learning and instruction explore the relationships between the determinants of learning and learning success as well as the causal mechanisms that mediate these relationships. Based on knowledge about these relationships and causal mechanisms, they develop guidelines that help to design learning environments and learning processes. To this end, researchers typically collect data with a multilevel structure in which individuals are nested within clusters; for example, students are nested within classrooms. Thus, multilevel modeling (MLM; e.g., Raudenbush & Bryk, 2002) is a prominent modeling approach because it allows researchers to account for this multilevel structure. One disadvantage of MLM is that it places high demands on the data. Estimation problems may occur and estimates of class-level (L2) regression coefficients may not be very accurate (accuracy problem) when sample sizes are too small. Therefore, Hox (2010) suggested sample sizes of 100 school classes be used. However, McNeish and Stapleton (2016) have noticed that researchers in the field of learning and instruction often ignore such recommendations.Extensive simulation work has revealed the sources of estimation problems: In small samples, covariance matrices at L2 tend to be degenerate, and thus, estimates of L2 regression coefficients tend to be rather inaccurate. Based on these findings, we speculate that approaches for stabilizing covariance matrices at L2 (i.e., regularization techniques) would be useful to mitigate such problems. Regularization techniques can be implemented with any kind of estimation approach (e.g., Bayesian estimation).With the proposed project, we will deepen the understanding of how these techniques work and thereby contribute to the development of approaches for estimating MLMs. Specifically, we will propose a method that allows for optimal estimation of MLMs in small samples. This method will yield estimates that are more accurate than any standard software. We will implement this method within the Maximum Likelihood, Bayesian, and factor score regression frameworks. In addition, we will make the optimized procedures available by publishing an R package, which will allow researchers to obtain more accurate estimates, particularly when sample sizes are small.

• DFG Programme: Research Grants