Human Manipulation Actions: Neurophysiological Validation of their Formal Characterization

• Applicants: Professorin Dr. Ricarda Ines Schubotz; Professor Dr. Florentin Wörgötter
• Subject Area: Human Cognitive and Systems Neuroscience ; Cognitive, Systems and Behavioural Neurobiology
• Term: from 2016 to 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 310431995

Human action is composed of chunks that are smoothly joined to form a continuous signal. Correspondingly, human observers as well as computers can decompose (segment) actions into characteristic phases. However, it remains unclear whether or not these phases are representative and relevant for action processing in the brain. To address this problem, we seek to quantify action phases asking whether purely data-driven action segmentation by computers is convergent with signatures of natural action segmentation as measured by human brain responses and behavior. In this project we focus on human object manipulation. Based on earlier works we will use machine vision methods to extract intrinsic properties of manipulations given by changes in object relations and trajectories. This way we can segment manipulations in an objective way, in the sense of being independent of overt human behavioral judgments. The first goal is to describe a large set of manipulations and to verify the hypothesis that the here-suggested data-driven segmentation into phases leads to a characteristic manipulation description that is independent of the individual way in which a manipulation has been performed.We then will assess to what degree brain activity and human behavior are linked to the derived phase structure. We address both conventional, overt human segmentation as well as covert segmentation by brain activity. Functional MRI BOLD attenuation is measured to assess the brain's expectation and surprise (i.e. information processing) during ongoing action observation, whereas Multi-Voxel Pattern Analysis will help to determine whether trajectory and object orientation are used by the brain to classify actions in the way the computational approach suggests. Thus, the common aim of the planned work is to arrive at an objective description of actions, and to biologically validate this approach to the characterization of human object manipulations.

• DFG Programme: Research Grants