SINIR – Simulating INteractive Information Retrieval

• Applicants: Professor Dr. Michael Granitzer; Professor Dr. Matthias Hagen; Professor Dr. Klaus Tochtermann
• Term: from 2018 to 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 408022022

Digital libraries today offer a wealth of their content through interfaces that users can interact with. In the backend of a digital library, sophisticated retrieval systems match a user's query to items from the digital library that are relevant to the query. However, access to digital libraries will never be static since the content is constantly updated, the retrieval functionality might be improved (e.g., by means of a software update), or the user interface might change. Major web search engines constantly implement subtle changes to their retrieval backend and user interface and evaluate their success using A/B-tests. Successful A/B testing requires a huge amount of user interactions, which are typically not available in digital library settings.Therefore, we suggest to simulate interactive user behavior based on cost/gain models and assumptions about typical user behavior. Simulated users representing an approximation of real behavior can be “generated” at any desired volume and variety such that A/B-testing for digital libraries becomes scalable. The overall goal of our project is to develop a simulation framework allowing the assessment of changes to a retrieval system and user interface of a digital library by analyzing the cost/gain impact on simulated users before implementing the changes in a real production system. In the SINIR project (Simulating INteractive Information Retrieval), we will develop an open-source framework that simulates different retrieval backend and interface configurations and evaluates their usefulness based on user models. The simulations comprise user interaction like querying, clicking, result inspection, etc. Some simulated users will be “trained” on real log data while other “artificially” generated users will follow some special assumptions (e.g., only clicking on relevant results). Based on the simulated users, also the meta-data usage of digital library content will be evaluated, potentially indicating improvements for subject indexing. The overall framework will be evaluated within a realistic digital library setting available in form of EconBiz at one of the project partners.The open-source simulation framework will enable digital libraries to test a much wider range of digital library system variants. Thus, the decisions about potential changes to the retrieval backend and interface can be based on a more thorough data basis, and changes can be rolled out quicker and more informed.

• DFG Programme: Research data and software (Scientific Library Services and Information Systems)
• Cooperation Partner: Professor Dr. Benno Stein
• Co-Investigators: Dr. Timo Borst; Dr. Tamara Pianos