Project Details
Projekt
Co-speech gestures and prosody as multimodal markers of information structure
Subject Area
General and Comparative Linguistics, Experimental Linguistics, Typology, Non-European Languages
Term
since 2022
Project identifier
Deutsche Forschungsgemeinschaft (DFG) - Project number 502013510
Information structure (IS) can be conceived of as a ‘cognitive domain’ interacting with the linguistic modules syntax, phonology, and morphology to control, update, and infer interlocutors’ common beliefs (Zimmermann & Féry 2010). While the interaction between IS and syntactic and phonological/prosodic patterns has been widely investigated, less is known about the visual marking of IS. From a prosodic standpoint, it is well-known that languages use prosodic features like pitch accentuation to convey IS (Calhoun & Kügler 2020 for a review). For example, in Germanic languages, given referents tend to be deaccented more often than new referents (Baumann & Grice 2006; Baumann & Schumacher 2020; Féry & Kügler 2008; Ladd 2008). By contrast, Romance languages have been reported to place focused constituents in phrase-final position (Zubizarreta 1998) and to lack systematic deaccentuation of given referential expressions (Ladd 2008; Swerts et al. 2002). From a multimodal perspective, it is a well-established fact that co-speech gestures typically align with pitch accented syllables across languages (McNeill 1992; Kendon 2004; Loehr 2004). Even though some gesture studies have hinted at the relationship between the presence of "beat gestures" and the marking of focused information, to our knowledge very few empirical investigations have assessed whether IS has an impact on (a) the presence of co-speech gestures; and (b) the gesture-speech timing patterns, as well as the prominence levels of the whole gesture.The main goal of this project is to assess the multimodal marking of IS from a crosslinguistic point of view. The two languages under investigation, Catalan and German, share the use of pitch accents for prosodic prominence marking of IS. They differ however in their positioning of accent patterns in relation to IS (variable focus position in German vs. tendency for rightmost focus position in the utterance in Catalan). Starting from the well-established coordination between co-speech gestures and prosodic prominence, the general question we ask is whether different information structural categories are signalled not only by prosodic events but also through the use of co-speech gestures. The project will use both controlled experimental materials as well as speech corpora of spontaneous discourse to analyse the interface between IS marking and prosodic and gestural features. We expect that in both Catalan and German (a) co-speech gestures will be associated more frequently with pitch accents that mark focus domains and novel information; (b) that the time alignment of co-speech gestures with speech will be directly affected by IS domains; and (c) that gesture amplitude patterns are directly correlated with degrees of prosodic prominence. In general, we expect that the presence of co-speech gesture will not be exclusively predicted by the presence/absence of pitch accentuation, but rather will have a unique role that will be predicted by IS.
DFG Programme
Priority Programmes
Subproject of
SPP 2392:
Visual communication. Theoretical, empirical, and applied perspectives (ViCom)
International Connection
Spain
