Resolving the neural basis of affective sound-meaning associations

• Applicant: Arash Aryani, Ph.D.
• Subject Area: Biological Psychology and Cognitive Neuroscience ; General, Cognitive and Mathematical Psychology
• Term: since 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 464844750

In contrast to the linguistic theory, a growing body of work has provided evidence that the sound of a word often carries subtle cues to its meaning, and that sound-meaning association in vocabulary is a general property of human language, which plays a crucial role for both phylogenetic language evolution and ontogenetic language development. Particularly in the affective domain, recent empirical results have shown that the sound of words can potentially convey emotional information, which can, in turn, i) play a mediating role in crossmodal correspondences (e.g., in the bouba-kiki-effect, where people tend to link words such as bouba to rounded shapes and kiki to spiky shapes), and ii) interact with the words’ semantic content (e.g., people give a higher arousal rating to the meaning of a word when its sound is harsh and arousing). The overarching goal of the proposed research is to resolve the largely unknown neural mechanisms, by which the brain extracts affective information from the sound of a word and links this information to both i) low-level sensory-perceptual information (e.g., visual shapes in a bouba-kiki-effect) and ii) high-level lexico-semantic information (e.g., affective meaning of a real word). This will be tackled in two work packages (WP1-2) accordingly. In detail, WP1 focuses on the bouba-kiki-effect to unravel the neural basis of emotional mediation in this matching bias. We will use multivariate pattern classification approach to show that the brain networks (fMRI) involved in matching words with shapes in a bouba-kiki task are similar to those engaged in linking other types of emotional auditory and visual stimuli to one another, e.g. linking environmental sounds to affectively congruent facial expressions. WP2 focuses on the processing of real words to provide a detailed account of the spatial (fMRI) and temporal (EEG) aspects of neural representations underlying the effect of sound on lexico-semantic processing in the affective domain. Here, we will comprehensively sample the words’ sound and meaning in the affective space (valence and arousal) and use representational similarity analysis (RSA) to resolve the mappings between neural activity and both the affective potential in the sound of words and its interaction with words’ lexical affective meaning. We will then combine fMRI and EEG data in a novel multivariate analysis framework to reveal the complex spatiotemporal neural dynamics underlying the effect of sound on meaning. In sum, the proposed research will provide a detailed neural account of affective sound-meaning associations at both levels of sensory-perceptual and lexico-semantic processing. The results have potential to shed light on how abstract concepts are grounded during language development by highlighting the role of emotion in non-arbitrary mappings, and have practical implications in improving current therapeutic strategies for individuals with aphasia as well as autism spectrum disorders.

• DFG Programme: Research Grants