Project Details
Projekt
AI-Enhanced Validation of Survey Instruments: Integrating Semi-Automated Methods in Cognitive Interviews for Children’s Self- and Proxy-Assessments of Health (AI-SIC)
Applicants
Dr. Jacqueline Kroh; Dr.-Ing. Andreas Niekler; Dr. Stephan Poppe; Professorin Dr. Julia Tuppat
Subject Area
Empirical Social Research
Term
since 2024
Project identifier
Deutsche Forschungsgemeinschaft (DFG) - Project number 539598673
The project investigates the potential of artificial intelligence (AI) and machine learning (ML) in coding open-ended, qualitative question formats in the context of quantitative, large-scale quantitative studies. The specific use case is cognitive interviews with children and parents on the question of how and on the basis of which underlying cognitive processes they assess the child's general state of health. The project thus addresses the use of novel AI-based technologies in the validation of a classical survey instrument. Instruments in survey research have traditionally been validated using either qualitative or quantitative methods. Qualitative approaches offer insights into how participants interpret and think about survey questions, while quantitative methods investigate correlations between concepts, for instance. However, this separation of approaches has resulted in fragmented research on instrument validity, with insufficient exploration of the implications for the scientific use of these instruments. By amalgamating the respective singular merits of qualitative and quantitative approaches, the comprehensive validation of survey instruments can be achieved, thus enhancing survey research at large. We aim to create a seamless collaboration of human expertise and machine efficiency by utilizing Artificial Intelligence within an Active Learning (AL) framework. Accordingly, this project seeks to establish a model of best-practice for integrating AI into qualitative research design coding, with a case study evaluating the validity of children's self-assessments of health. The research questions it addresses are as follows: How can AI and Machine Learning improve qualitative research designs such as cognitive interviews? What role can Machine Learning play in validating health assessments given by children, adolescents, and parents? The InTraCo Framework (Integrated Transformer-based Inductive Coding of Language Utterances with AL) will be developed, implemented and validated in the initial stage of our project. This framework tackles challenges such as genuine multi-label classification and the incorporation of multiple coders. We investigate the effectiveness of semi-automated coding in real-world scenarios, the impact of human-machine interaction defined in InTraCo on the coding process, the levels of precision and accuracy achieved by our Encoder-Decoder model through AL for the proposed research task, and the insights that can be gained from explainable AI methods applied to the final trained model. In the latter part of the study, we examine the benefits of our method utilizing explainable AI techniques, and scrutinize the cognitive processes underlying the self- and proxy- assessment of child health. We will analyze the cognitive processes of parents and children when assessing the health of children systematically, whilst considering qualitative and quantitative perspectives, consequently tackling pertinent areas within research.
DFG Programme
Infrastructure Priority Programmes
