Using electrocorticography to investigate the role of neuronal oscillations in multisensory speech perception
Final Report Abstract
Human speech contains both auditory and visual components, processed by their respective sensory cortices. We test a simple model in which task-relevant speech information is enhanced during cortical processing. Visual speech is most important when the auditory component is uninformative. Therefore, the model predicts that visual cortex responses should be enhanced to visual-only speech compared to audiovisual speech. We recorded neuronal activity as patients perceived auditory-only, visual-only and audiovisual speech. Visual cortex showed strong increases in high gamma-band power and strong decreases in alpha-band power to visual-only and audiovisual speech. Consistent with the model prediction, gamma-band increases and alpha-band decreases were stronger for visual-only speech. The model predicts that the uninformative nature of the auditory component (not simply its absence) is the critical factor, a prediction we tested in a second experiment in which visual speech was paired with auditory white noise. As predicted, visual speech with auditory noise showed enhanced visual cortex responses relative to audiovisual speech. An examination of the anatomical locus of the effects showed that all visual areas, including primary visual cortex, showed enhanced responses. Visual cortex responses to speech are enhanced under circumstances when visual information is most important for comprehension.
Publications
- (2012) Auditory speech diminishes gamma-band responses in visual cortex to visual speech. "Society for Neuroscience Meeting", 13.-17. October 2012, New Orleans, USA
Schepers, I.M., Sun, P., Pei, X., Baum, Yoshor, D., Beauchamp, M.S.
- (2012) Creating visual percepts with electrical stimulation of human visual cortex: comparison with receptive fields mapped with local field potentials. "Society for Neuroscience Meeting", 13.-17. October 2012, New Orleans, USA
Pei, X., Sun, P., Schepers, I.M., Beauchamp, M.S., Yoshor, D.
- (2013) Diminished high gamma response in superior temporal sulcus during incongruent audiovisual speech perception. "Society for Neuroscience Meeting", 9.-13. November 2013, San Diego, USA
Ozker, M., Schepers I.M., Yoshor D., Beauchamp M. S.
- (2013) Electrocorticography of visual cortex responses to multisensory speech. "Annual Meeting of the Organization for Human Brain Mapping ", 16.- 20. June 2013, Seattle, USA
Schepers, I.M., Sun, P., Pei, X., Baum, Yoshor, D., Beauchamp, M.S.
- (2013). Noise alters beta-band activity in superior temporal cortex during audiovisual speech processing. Neuroimage, 70:101-112
Schepers, I. M., Schneider, T. R., Hipp, J. F., Engel, A.K., & Senkowski, D.
(See online at https://doi.org/10.1016/j.neuroimage.2012.11.066)
