Learning foreign language with visual and motor enrichment - Learning outcomes and neurobiological mechanisms

• Applicant: Professorin Dr. Katharina von Kriegstein
• Subject Area: Human Cognitive and Systems Neuroscience
• Term: from 2013 to 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 243364146

The presence of information from multiple sensory modalities during learning, known as learning enrichment, has been shown to benefit learning outcomes in many domains, including voice identity recognition, music training, and foreign language instruction. For example, the learning of auditory foreign language vocabulary paired with one’s own body movements or with pictures enhances memory for that vocabulary compared to auditory-only learning. Mechanisms underlying the benefits of learning enrichment are currently unknown. Our primary goal is to discover key behavioural and neuroscientific principles underlying learning enrichment in the domain of foreign language vocabulary learning. Our hypotheses and experimental approach are driven by the multisensory learning theory, which attributes enrichment benefits to the reactivation of brain areas that process the enrichment during learning. For example, enrichment with self-performed gestures would lead to reactivation of biological movement visual areas and motor cortices during later auditory-only translation tasks. Hypotheses derived from the multisensory learning theory necessitate a multi-methodological approach to test (i) whether brain regions that are active during learning become reactivated during subsequent vocabulary translation, (ii) whether the reactivation occurs at a time point before vocabulary translation is complete, and (iii) whether the responses in the reactivated areas are causally relevant for learning outcomes. In the initial phase of the grant we used systematic behavioural tests, spatially highly-resolved functional magnetic resonance imaging (fMRI), and transcranial magnetic stimulation (TMS) of visual movement brain areas. The findings of these studies supported the multisensory learning theory rather than opposing theories (e.g., depth of processing theories). In the next phase of the grant, we plan to perform temporally highly-resolved magnetoencephalography (MEG) and TMS studies investigating the role of the motor cortices in enriched learning. The involvement of motor cortices (in addition to specific visual cortices) may explain an intriguing behavioural finding from the initial phase of the grant: Enrichment of foreign language vocabulary learning by performing gestures is even more powerful than enriching learning by viewing pictures in adults. The proposed studies will advance our knowledge of how the human brain optimally learns foreign language vocabulary. We expect that the findings will bridge two currently separate influential frameworks in neuroscience: the views that motor processes or visual processes support the understanding of spoken language. The proposed research also has application potential. We plan to take initial steps toward evaluating benefits of enriched learning in the context of foreign language teaching in schools; the expected findings will pave the way towards the development of evidence-based foreign language teaching techniques.

• DFG Programme: Research Grants
• International Connection: Austria
• Co-Investigators: Dr. Manuela Macedonia; Dr. Brian Mathias