The grounding of abstract concepts in vision and action
Human Cognitive and Systems Neuroscience
Final Report Abstract
Concepts held in semantic long-term memory are basic units of cognition and support object recognition, action planning, problem solving and verbal communication. Concepts refer to concrete objects (“table”) or actions (“to knock”), but also have referents, which cannot be perceived or acted upon, like mental or emotional states (“hope”), social constellations (“cooperation”), abstract ideas (“justice”), and scientific phenomena (“gravity”). The latter types of concepts, which do not refer to entities for which a direct sensory-motor experience is available, are traditionally termed “abstract concepts”. In particular, knowledge representation of concepts with an abstract referent such as mental states is matter of a fierce current debate. The representation of abstract and concrete concepts can be quite naturally handled by amodal models. According to these traditional models in the cognitive sciences, concepts are viewed as amodal symbolic mental entities, different from the perceptual or motor brain systems. In contrast, more recent grounded (or embodied) cognition theories propose that distributed modal experiential representations of external (perception) and internal states (proprioception, emotion and introspection) as well as actions establish conceptual representations. In a refinement of this approach with regard to abstract concepts, we propose that abstract concept representation is heterogeneous and depends on a variety of experiential channels like emotions, introspections, mental states, action and perception, complemented by verbal associations. In the present project, we tested prediction derived from grounded theory by assessing functional and anatomical characteristics of the processing of abstract concepts. In 11 studies, their semantic content, the precise functional-anatomical localization and time course of brain activity, the functional relevance of the sensory-motor systems during conceptual processing as well as plastic changes in conceptual processing in experts were investigated by functional magnetic resonance imaging, event-related brain potential recordings and behavioral measurements. In a first step, the semantic content of a large set of 296 abstract concepts was determined in a property listing study. Participants generated a substantial proportion of sensory-motor, introspective, emotional and social properties, in addition to verbal associations. Cluster analyses of the generated properties demonstrated several subgroups of abstract concepts characterized by different modal property types. Subsequent event-related potential and imaging studies showed that abstract concepts with a strong relevance of visual, motor and mental state features are processed in corresponding modal cortex with an early onset of activity. Furthermore, a behavioral study revealed the functional relevance of the visual system for abstract concept processing. Finally, in line with grounded cognition theories we showed in an expertise study in psychotherapists and undergraduate psychology students on clinical-psychological concepts that even highly abstract scientific concepts are grounded in modal brain systems and are experience-driven. We observed a stronger grounding of information related to social constellations in the mentalizing system in psychotherapists compared to students. At a methodological level, our research highlights the importance of investigating the processing of sets of abstract concepts with a clearly defined semantic content. The results of our project thus considerably contributed to establishing grounded cognition theories as a general approach to explain the representation of both concrete and abstract concepts. Based on our findings, we propose a refined grounded cognition theory, which explains the representation of abstract concepts by an interaction between modalityspecific, multimodal, language and amodal hub areas. The notion of grounding of conceptual knowledge is highly relevant for improving education, but also for art and clinical psychology. We therefore presented research related to this project at conferences or public talks within these fields.
Publications
- (2018). The semantic content of abstract concepts: A property listing study of 296 abstract words. Frontiers in Psychology, 9:1748
Harpaintner, M., Trumpp, N. M., & Kiefer, M.
(See online at https://doi.org/10.3389/fpsyg.2018.01748) - (2020). The grounding of abstract concepts in the motor and visual system: An fMRI study. Cortex, 124
Harpaintner, M., Sim, E.-J., Trumpp, N. M., Ulrich, M. & Kiefer, M.
(See online at https://doi.org/10.1016/j.cortex.2019.10.014) - (2020). Time course of brain activity during the processing of motor- and vision-related abstract concepts: Flexibility and task-dependency. Psychological Research
Harpaintner, M., Trumpp, N. M., & Kiefer, M.
(See online at https://doi.org/10.1007/s00426-020-01374-5) - (2020). Varieties of abstract concepts and their grounding in perception or action. Open Psychology, 2, 119-137
Kiefer, M., & Harpaintner, M.
(See online at https://doi.org/10.1515/psych-2020-0104)
