Automated question generation with feedback for stimulating reflections on code quality

• Applicants: Professor Dr. Sven Strickroth; Professor Dr. Michael Striewe
• Subject Area: Software Engineering and Programming Languages
• Term: since 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 528518019

Software quality is an important topic both in software engineering research and in practice. There are several software quality attributes, among of which functional correctness is often assumed the most important one as it determines whether a piece of software is useful at all. Other attributes such as robustness, performance, maintainability, and comprehensibility are nevertheless also of great importance for the success of a software engineering endeavor. In contrast to that, novices in programming often only focus on functional correctness and therefore the code often has a low code quality. A reason for this might be that common automatic training or grading tools focus on correctness only or provide (simple) feedback based on software metrics that might not be understandable and actionable for learners. Although recent research shows that teachers struggle with teaching code quality in general, there are didactic concepts available that embrace code quality. However, these approaches do not scale well for large classes and do not allow to stimulate a discussion on code fragments from every student. There are also approaches to provide feedback on coding style at scale, but these are not concerned with more diverse aspects of code quality like security, robustness, or performance. Hence, a key problem apparently is not a lack of ideas for instructional design and pedagogical setup, but a lack of fundamental concepts on how to stimulate discussions on diverse aspects of code quality in an automated, scalable way, similar to the well established automated feedback on functional correctness. This gap between the great relevance of code quality in practical application and the low availability of fundamental concepts on how to provide actionable feedback to learners or also to professional developers in an automated, scalable way is the starting point for this proposal. The goal is to automatically generate questions for individual student submissions that cover different dimensions as outlined above. The focus of the project is not about pedagogical questions, such as how and when to discuss code quality during classroom or exercise sessions, but on the concepts and mechanisms that are required to include discussion and reflection on code quality into automated training and assessment systems for programming. This includes both the automated generation of appropriate, individual questions as well as the automated provision of individual feedback to the answers. The latter is important to make sure the prompts for reflection and discussion are actually used and understood by the students. Thus, the overarching research question is how to automatically generate questions that stimulate reflection and discussion on code quality and evaluate the answers -- both based on the code submitted by the students, and in a scalable way.

• DFG Programme: Research Grants