Bayesian Approaches to Preference-based Answer Generation in Dialogue
Final Report Abstract
The projects were motivated by our view that game-theoretic models of communication benefit from a more elaborated handling of the speaker’s tasks, and answer generation in a dialogue system will benefit from game-theoretic models. A prospective tenant asks questions about apartments the system as a real estate agent is offering. The system answers these questions directly and indirectly, respectively. For example, the question: Does the apartment have a garden? could be answered by the system directly, but it is also able to generate the exemplary answers It has a balcony or If you are interested in growing flowers, it has a balcony. Furthermore, if the customer asks Is there a supermarket? the system can choose between answers that convey vague or exact distances as, for example, There is a supermarket fewer than 200 / 170 / more than 150 m away. The underlying research question is how the system can make optimal strategic use of such indirect answers. Most game theoretic models of communication are based on signaling games, i.e. games in which first the speaker chooses an utterance from a range of alternatives, and the hearer then picks an interpretation. Then the game ends. We considered this inadequate as a model of sales scenarios. If the customer asks Does the house have a garden?, the dialogue does not simply end when the system answers No. If the customer wants a place for growing flowers, it is likely that s/he will next ask for a balcony. The system can anticipate this question and answer to the first question Well, it has a balcony instead of No. Indirect answers come with extra costs, because they are longer, but also save costs as they lead to overall shorter dialogues. This scenario can be described as a strategic situation in which the system has to balance costs and savings of indirect answers. We had to develop an explicit game-theoretic model of this situation, learn probabilities of possible follow up questions from data, determine a threshold that decides when indirect answers are profitable, and evaluate the system that was based this model. This approach was very successful and culminated in two noteworthy publications. The follow-up project extended the linguistic domain. We integrated speech act conditionals and modified numerals as (part of) answers to the dialogue system, which required a corresponding extension of the non-linguistic domain and the assumed requirements. As before, the probabilistically motivated assumptions have been underpinned by experimental studies carried out via the crowdsourcing platforms MTurk and Prolific, and the system behavior had been evaluated as before. In general, we found that indirect answers support the user’s decision for or against an apartment offered by the system as sales agent, since the user’s requirements, expressed by the various forms of indirect answer we have examined in our projects, reduce the number of questions that must be raised in order to come to a decision. From a theoretical perspective, we were able to demonstrate the benefits of using concepts from game theory for computational models of answer generation.
Publications
- 2014. Indirect answers as potential solutions to decision problems. In Verena Rieser and Philippe Muller (eds.), Proceedings of DialWatt - SemDial 2014, 145-153. Edinburgh: U Edinburgh
Stevens, J., A. Benz, S. Reuße, R. Laarmann-Quante and R. Klabunde
- 2015. A strategic reasoning model for generating alternative answers. Proceedings of ACL 2015, 534-542. Beijing: Association for Computational Linguistics
Stevens, J., A. Benz, S. Reuße and R. Klabunde
(See online at https://doi.org/10.3115/v1/P15-1052) - 2015. Pragmatic query answering: Results from a quantitative evaluation. In Chris Biemann, Siegfried Handschuh, André Freitas, Farid Meziane & Elisabeth Metais (eds.), Lecture Notes in Computer Science, Natural Language Processing and Information Systems: Proceedings of NLDB 2015, 110-123. Berlin: Springer
Stevens, J., A. Benz, S. Reuße and R. Klabunde
(See online at https://doi.org/10.1007/978-3-319-19581-0_9) - 2015. The role of preferred outcomes in determining implicit questioning strategies. Proceedings of the 20th Amsterdam Colloquium. 378-387
Stevens, J.
- 2016. Generating Surplus Content in a Q/A- Setting. Proceedings of SemDial 2016
Reuße, S. R. Klabunde, J. Stevens and A. Benz
- 2016. Pragmatic question answering: A gametheoretic approach. Data & Knowledge Engineering 106
Stevens, J., A. Benz, S. Reuße and R. Klabunde
(See online at https://doi.org/10.1016/j.datak.2016.06.002) - 2016. When do we think strategically? Zeitschrift für Sprachwissenschaft 35(1), 97-108
Stevens, J.
(See online at https://doi.org/10.1515/zfs-2016-0007) - 2018. A Case Study on the Relevance of the Competence Assumption for Implicature Calculation in Dialogue Systems. In: Rehm, G. and Declerck, T. (Eds.) Language Technologies for the Challenges of the Digital Age. 27th International Conference, GSCL 2017. Berlin: Springer Open; 276-283
Fischer, J.
(See online at https://doi.org/10.1007/978-3-319-73706-5_23) - 2018. Communicating an understanding of intention: Speech act conditionals and modified numerals in a Q/A system. In Proceedings of SemDial 18, pp. 103–111
C. Hesse, M. Mohammadi, J. Fischer, M. Langner, A. Benz, and R. Klabunde
- 2018. Conveying the user’s intention by generating speech act conditionals as indirect answers. Proceedings of KONVENS 2018, 80-88
M. Mohammadi, J. Fischer, M. Langner and R. Klabunde
- 2018. Game-Theoretic Approaches to Pragmatics. Annual Review of Linguistics 4, 173-191
Benz, A. and J. Stevens
(See online at https://doi.org/10.1146/annurev-linguistics-011817-045641) - 2020. Scalar bounds and expected values of comparatively modified numerals. Journal of Memory and Language, 111
C. Hesse and A. Benz
(See online at https://doi.org/10.1016/j.jml.2019.104068)
