The interaction of memory and reinforcement learning in value-based choice

• Applicant: Professor Dr. Radoslaw Martin Cichy, since 7/2022
• Subject Area: Human Cognitive and Systems Neuroscience
• Term: from 2018 to 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 412917403

To make successful decisions in dynamic environments, human decision makers must adaptively change the values they attribute to different choice options. Many value-based decisions rely on long-term memory. Nevertheless, decision making has generally been investigated in isolation from long-term memory. Recent studies suggest that interactions between the hippocampus (HC), ventral striatum (VST), and ventromedial prefrontal cortex (VMPFC) are crucial for memory-guided decisions. Importantly, these brain regions mature at different rates and show substantial individual differences in structure and function. It is currently not known how maturational changes of relevant brain regions from middle childhood to early adulthood as well as individual differences in adulthood alter the interaction between memory and decision-making systems. The first aim of this project is to neurocomputationally dissect the contribution of the HC, VST, and VMPFC, respectively, to memory-based, value-guided decisions. Based on this dissection, the second aim is to develop a mechanistic model of the extent and direction of functional connections among the HC, VST, and VMPFC when making memory-based, value-guided decisions. This model will then be used to formally capture how individual differences in brain structure and function among younger adults contribute to individual differences in decision making. Finally, the third aim of the project is to examine how maturational brain changes from middle childhood to early adulthood modulate memory-valuation interactions. In pursuit of these goals, we will combine a multi-modal, model-based, neuroimaging approach with a neurodevelopmental perspective.In the first work package (WP1), we will neurocomputationally dissect the contribution of the HC, VST, and VMPFC, respectively, in realizing associative memory-guided, value-based decision-making in adults. To this end we will use a model-based fMRI approach, combining fMRI experiments in young adults performing a memory-guided, value-based decision-making task, and modeling behavioral data using reinforcement learning. Next, we will test the extent and direction of the connections between the HC, VST, and VMPFC in realizing associative memory-guided, value-based decision making using dynamic causal modeling (DCM). In the second WP, we will test how individual differences in brain structure (DTI), brain function (fMRI), and behavior (accuracy, RT, participant specific model parameter estimates) are related to each other during value updating and memory-guided decision making. In the third WP, we will adapt the experimental paradigm used in WP1 and WP2 for children aged 8–10 years and adolescents aged 13–15 years. Then we will use model-based fMRI, as well as DTI, to examine the degree to which age-related, differences in structural (DTI) and functional connectivity (fMRI) are associated with age differences in memory-guided, value-based decision making.

• DFG Programme: Research Grants
• Co-Investigator: Professor Ulman Lindenberger, Ph.D.
• Ehemaliger Antragsteller: Professor Dr. Hauke Reiner Heekeren, until 6/2022