This project investigates the influence of motor learning mediated by different sensory systems (purely auditory or purely visual) on the auditory or visual perception of movements, the crossmodal interactions and its neural processing (especially within the structures of the mirror neuron system).
Final Report Abstract
1) Cross-modal transfer: In order to study cross-modal transfer between modalities, non-musicians learned to play short sequences on a piano keyboard in a audio-motor training condition (merely listening and replaying while the vision of own fingers was occluded) or visuo-motor training condition (merely observing silent finger movements and replaying on a silent keyboard). After training, participants recognized the vision of finger movements corresponding to audio-motor trained melodies more accurately, more rapidly, and more confidently as having trained than the corresponding sound of visuo-motor trained sequences. fMRI revealed that observing finger movements corresponding to audio-motor trained melodies is associated with stronger activation in the left rolandic operculum than observing untrained sequences. This region was also involved in silent execution of finger movement sequences, proposing that a link to motor representations may play a role in cross-modal transfer from the audio-motor training condition to visual recognition. In contrast, no significant differences in brain activation were measured between listening to the corresponding sound of visuo-motor trained compared to untrained sequences. Thus, neural representations generated in the visuo-motor training sessions did not (yet) become available enough to the auditory modality to generate significant differential brain activation. That Cross-modal transfer was stronger from audio-motor training condition to visual recognition might be related to the fact that non-musicians are familiar with how their finger movements look (motor-to-vision transformation), but not with how they sound on a piano (motor-tosound transformation). 2) Interindividual differences and relations to properties of white matter fiber tracts: Non-musicians learned to perform three short melodies on a piano keyboard in a pure audio-motor training (vision of their own fingers was occluded). Initial learning times varied substantially and ranged from 17 to 120 minutes (mean±SD: 62±29 minutes). Participants were divided into equal groups of fast and slow learners. Diffusion-weighted magnetic resonance imaging was used to derive the fractional anisotropy (FA), an index of white matter microstructural arrangement. Group comparisons revealed higher FA values for fast than slow learners, and correlation analyses indicated that higher FA values were associated with faster learning for the whole sample. These effects were observed in the corticospinal tract, a bundle of axons relevant for the execution of voluntary movements, and the superior longitudinal fasciculus, a tract important for audio-motor transformations. These results suggest that the speed with which novel complex sensorimotor skills can be acquired may be determined by variability in structural properties of white matter fiber tracts connecting brain areas functionally relevant for motor learning. 3) Cross-modal adaptation: A repetition suppression (RS) paradigm was used in order to investigate multisensory properties of brain areas (audio-visuo-motor properties). Participants learned to play six piano sequences in a motor training condition that included listening to the sounds of melodies and observing corresponding finger movements. Subsequently their brain activity was recorded during perception of these trained and additional non-trained sequences. Participants first listened to a melody and subsequently observed finger movements of the same sequence (repeats) or a different sequence (non-repeats). Comparing non-repeats vs. repeats, suppression of the hemodynamic response was stronger for trained compared to non-trained sequences in the left BA44 and the bilateral dorsal premotor area (dPM). This RS effect might be caused by an extraction of the corresponding motor pattern and its sequential information during both, listening to a melody and observation of finger movements. These results support the assumption of multisensory properties in BA44 and the dPM.
Publications
- (2010) Crossmodal repetition suppression for auditorily and visually perceived actions. 11th International Multisensory Research Forum, Liverpool, United Kingdom, 16-19 June, 2010. p43
Hijmans, B.S., Engel, A., Bangert, M., Keller, P.E., Keysers, C.
- (2010) Crossmodal transfer for newly learned finger movements on a piano keyboard. 11th International Multisensory Research Forum, Liverpool, United Kingdom, 16-19 June, 2010. p.76
Engel, A., Bangert, M., Hijmans, B.S., Horbank, D., Keller, P.E., Keysers, C.
- (2010) Inter-individual differences in motor learning and white matter fiber tract architecture. International Students Congress of Medical Sciencs, Groningen, Netherlands, 08-11 June, 2010
Hijmans, B.S., Engel, A., Cerliani, L., Bangert, M., Nanetti, L., Keller, P.E., Keysers, C.
- (2011). Inter-individual differences in auditory learning of piano sequences and white matter fiber tract architecture. Rhythm Perception and Production Workshop, Leipzig, Germany, 13-15 July, 2011
Engel, A., Hijmans, B. S., Cerliani, L. Keller, P. E., Bangert, M., Keysers, C.
- (2011). Interactions of perception and action in musical tasks: Detection of musical spontaneity and crossmodal transfer. Neuromusik Gesprächskreis. Max Planck Institute for Human Cognitive and Neurosciences, Leipzig, Germany. 03 March 2011
Engel, A.
- (2011). Learning Piano Melodies - in merely visual or auditory training conditions - and the neural correlates of their crossmodal recognition. Neuroscience & Vision Symposium. Netherlands Institute of Neurosciences, Amsterdam, The Netherlands. 18 February 2011
Engel, A.
- (2011). Learning piano melodies under visual or auditory training conditions and the neural correlates of their cross-modal recognition. The Neurosciences and Music IV: Learning and Memory, University of Edinburgh, Edinburgh, United Kingdom, 09-12 June, 2011
Engel, A., Bangert, M., Keller, P.E., Hijmans, B.S., Horbank, D., Wilkens, K., Keysers, C.
- (2011). Synchronized drumming enhances activity in the caudate and facilitates prosocial commitment - if the rhythm comes easily. PlosOne, 6(11), e27272
Kokal, I., Engel, A., Kirschner, S., & Keysers, C.
- Learning piano melodies in visuo-motor or audiomotor training conditions and the neural correlates of their cross-modal transfer. NeuroImage
Engel, A., Bangert, M., Horbank, D., Hijmans, B. S., Wilkens, K., Keller, P. E., Keysers, C.
(See online at https://doi.org/10.1016/j.neuroimage.2012.03.038)
