Risk-sensitive decision making under inclomplete information
Final Report Abstract
The aim of the project was to develop methods for modeling the behavior of agents that exhibit risk preferences when decisions must be made under partial information of the environment. During the time of the project, we obtained the following results. In "A Fenchel-Moreau-Rockafellar Type Theorem on the Kantorovich-Wasserstein Space with Applications in Partially Observable Markov Decision Processes" we extend an algorithm for solving risk neutral POMDPs from the nite state/action case to the case where the state, observation, and action spaces are continuous. Our main theoretical contribution is summarized in the manuscript "Risk-Sensitive Partially Observable Markov Decision Processes as Fully Observable Multivariate Utility Optimization Problems", where we treated models that combine risk-sensitivity, which is induced by utility functions, with partial observability of the enviroment. More specically, we managed to extend methods that require single exponential utility functions to treat utility functions that are weighted sums of exponentials. Since every increasing function can be approximated with any precision by a weighted sum of exponentials, our method can be used to treat utility functions of any risk preference. This includes utility functions which induce mixed preferences similar to what have been previously described to underlie human decision making under risk. In "Risk Sensitivity under Partially Observable Markov Decision Processes" we conducted behavioral experiments involving choice making, where we tried to explore if models that include risk preferences can be used to quantify human behavior in a POMDP setting. The goal of this study was to demonstrate (1) that risk-sensitivity of subjects can be quantied using utility functions also in situations where state information isincomplete and (2) that the amount of data generated in a typical behavioral experiment is suffcient to validate models of adequate complexity. Partial observability was introduced by using random dot kinematograms with low coherence as visual indicators of state. Fitting risk-neutral and risk-sensitive models to the subjects' behavioral response times we could identify subject groups with similar risk-preferences and we could show that the risk-sensitive models tted the experimental data considerably better than the risk-neutral model. This provides a proof-of-principle for the applicability of the investigated framework in a practical setting.
Publications
- (2019). A Fenchel-Moreau-Rockafellar Type Theorem on the Kantorovich-Wasserstein Space with Applications in Partially Observable Markov Decision Processes. J. Math. Anal. Appl. 477:1133-1156
Y. Shen, V. Laschos, W. Stannat, and K. Obermayer
(See online at https://doi.org/10.1016/j.jmaa.2019.05.004) - (2019). Risk Sensitivity under Partially Observable Markov Decision Processes. In: Proceedings of the Conference on Cognitive Computational Neuroscience 2019
N. Höft, R. Guo, V. Laschos, S. Jeung, D. Ostwald, and K. Obermayer
(See online at https://doi.org/10.32470/CCN.2019.1160-0) - (2022). Risk-Sensitive Partially Observable Markov Decision Processes as Fully Observable Multivariate Utility Optimization Problems
A. Afsardeir, A. Kapetanis, V. Laschos, and K. Obermayer
(See online at https://doi.org/10.48550/arXiv.1808.04478)
