Longitudinal Assessment of Students’ Learning in Educational Contexts: Dimensionality of Change and Change of Dimensionality

• Applicant: Dr. Alexander Naumann
• Subject Area: General and Domain-Specific Teaching and Learning
• Term: from 2018 to 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 407454729

Assessments of students’ competencies and achievement are widely used within educational contexts. Typically, teachers, researchers and policy-makers hope to retrieve useful information for pedagogical, scientific or political decisions from students’ test scores. For instance, in-class exams or admission tests are commonly applied tools for assessing individual students’ competencies or achievement. On the aggregate level, test scores in educational assessments have become a major criterion for determining the success or failure of teaching and schooling in Germany. Then, teachers, researchers and policy-makers have to deal with the challenge that the achievement test data are gathered at the individual level (i.e., students), which is then used to draw inferences on characteristics of the aggregate level (i.e., classes, courses, schools or school systems). Yet, while educational research puts much effort in fostering valid measurements of cluster-level characteristics using individual-level data, comparably less effort has been put into the meaning of achievement measures at the cluster-level so far. That is, whether the same dimensional structure of ability holds at the different levels (individual, classroom, etc.) or at different points in time is far less frequently investigated for achievement measures than for questionnaire measures.In particular, there is little knowledge on the dimensional structure with respect to repeated measurements of student ability at the level of schools or classrooms. Hypothetically, on the one hand, cluster-level change may be multidimensional and, on the other hand, there may be a change in cluster-level dimensionality between testing occasions. If multidimensionality at the cluster-level actually was found in repeated measurements of student achievement, not only the assumption of unidimensionality would be violated, but treating achievement as unidimensional would also discard information potentially relevant to practice, research or policy.Thus, the proposed project aims at developing a measurement framework for modeling repeated measurements of student achievement at the cluster level. This framework will allow to critically examine assumptions of dimensionality of change, change of dimensionality, or measurement invariance, respectively. The underlying question is whether a single value is appropriate for describing learning at the level of classrooms or schools. To achieve this aim, the project addresses three major objectives: (1) The provision of a model to evaluate multidimensional structures on the cluster level, (2) the investigation of the factor structure at the cluster level within empirical longitudinal achievement test data, and (3) the provision of an R package comprising screening procedures and tutorials adequate for practitioners.

• DFG Programme: Research Grants
• International Connection: Switzerland
• Co-Investigator: Professor Dr. Johannes Hartig
• Cooperation Partner: Professor Dr. Jan Hochweber