Project Details
Projekt
Ensuring valid comparisons of self-reports in heterogeneous populations and marginalised groups (ENSURE)
Applicant
Professorin Dr. Natalja Menold
Subject Area
Empirical Social Research
Public Health, Healthcare Research, Social and Occupational Medicine
Public Health, Healthcare Research, Social and Occupational Medicine
Term
from 2019 to 2023
Project identifier
Deutsche Forschungsgemeinschaft (DFG) - Project number 409654512
Many sub-projects in the research unit PH-LENS rely on data collection via surveys. When refugees and other marginalised populations are surveyed, data collection instruments should be adapted for use in these groups. According to the quality standards of the German Data Forum (RatSWD), high reliability and validity should be ensured when developing new or adapting existing measurement instruments. In comparative surveys, measurement invariance – a prerequisite for any between-group comparisons – should be additionally taken into account, which means that group membership should not bias the results of comparisons. Establishing measurement invariance in survey data has been a methodological challenge in cross-cultural surveys. Different ex-ante approaches and ex-post data analysis methods have been used to reach and evaluate comparability. One method is cognitive pretesting, which has been used to establish higher measurement quality and cross-cultural comparability during the piloting of questions. Another method is incorporating anchored vignettes that allow an ex-post adjustment of the data taking different usage of answer options into account. Correcting for response sets is a potential alternative to anchored vignettes. However, empirical evidence is lacking on whether these methods are effective in establishing measurement invariance and can improve measurement quality. The first objective is to ensure a high quality of measurement (reliability and validity) and measurement invariance of adaptations of instruments relevant to the PH-LENS. In close bi-directional work with the sub-projects NEXUS, LARGE, OTHER and TREAT, measurement instruments will be adapted, cognitively tested, and improved. Measurement quality and measurement invariance of adapted instruments will be assessed.The second objective is to evaluate procedures for developing measurement-invariant instruments of high reliability and validity. This part focuses on a comparison of cross-cultural cognitive pretests as an ex-ante method with ex-post data corrections when using anchored vignettes and correcting for response sets. To compare the methods, an experimental study on refugees in a large state (Baden-Wuerttemberg) will be conducted, sharing resources with the sub-projects NEXUS and TREAT.The third objective is to evaluate statistical methods of ex-post-factum analysis of measurement invariance (Bayesian SEM, alignment method, exact measurement invariance) with respect to their sensitivity in detection of problems of comparability known from cognitive pretests.The sub-projects of the PH-LENS benefit from the quality assurance of the measurement instruments as well as from the adapted, evaluated and documented high-quality measurement instruments for use in heterogeneous populations and marginalised groups. High-quality measurement instruments and methods to reach measurement invariance will also benefit further areas of health, behavioural and social science research.
DFG Programme
Research Units
