Neural mechanisms of real-world visual categorical decisions

• Applicants: Professor Dr. Radoslaw Martin Cichy; Professor Dr. Klaus Obermayer
• Subject Area: Human Cognitive and Systems Neuroscience
• Term: from 2019 to 2024
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 408187675

To survive, humans must make correct perceptual decisions. For this, their brains must form accurate representations of the world that can guide adaptive choice behavior. The overarching goal of the proposed research is to resolve the neural mechanisms by which the brain makes perceptual decisions about complex real-world visual stimuli. To reach this goal we will go methodologically beyond standard univariate analysis approaches and use a multivariate approach called the “distance-to-hyperplane approach”.The basic idea is to extend classical signal detection theory from a one-dimensional (univariate) to a multidimensional scenario such that the relation between multivariaten brain patterns and choice behavior can be assessed. This is done by using machine learning techniques that estimate a decision hyperplane in multivariate space, analogous to the concept of a criterion in one-dimensional space. As in the univariate case, in the multivariate case the distance to the /hyperplane predicts classification behavior: the greater the distance of the brain pattern evoked by a particular stimulus to the hyperplane, the stronger the evidence is, and the shorter the reaction time becomes (under the common assumption that evaluation of evidence is faster in this case). Within this framework, we will address two key aims that in one work package each. The first aim is to resolve the spatio-temporal neural dynamics that enables choice behaviour. In work package 1, we will 1) assess human categorization behavior (reaction times) for complex visual real-world scene stimuli, 2) record fMRI and EEG data and analyze them using multivariate pattern analysis methods to resolve the neural dynamics during perception with high spatial and temporal resolution, and 3) using the distance-to-hyperplane approach link the measurements of brain activity to behaviour.Work package 2 will address the ubiquitous ambiguity in perceptual decisions whether real-world visual stimuli are based on simple or complex visual features that co-occur and equally enable decisions, as represented in early (V1–V3) and late (IT) stages of the neural visual processing hierarchy. To assess the role of simple and complex visual features independently, we will create a stimulus set with a fully controlled visual feature content using high-throughput screening in combination with artificial deep neural networks. Using this stimulus set, we will investigate the role of each stimulus feature type on behavior, and whether behavior changes depending on depth of processing. In sum, the results of our project will provide a novel, detailed, and ecologically valid account of how the brain mediates perceptual decisions about complex visual scene stimuli.

• DFG Programme: Research Grants