Music4u: Personalized Objective Deep Learning Models to Make Music More Accessible for Cochlear Implant Users

• Applicant: Professor Dr.-Ing. Waldo Nogueira
• Subject Area: Otolaryngology, Phoniatrics and Audiology ; Medical Physics, Biomedical Technology
• Term: since 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 446611346

Music plays an essential role in people’s lives and is part of many socio-cultural and educational events. Music is the most complex acoustic signal as it uses the full dynamic, bandwidth, and resolution of the human auditory system. Moreover, music is part of being human and connects people through emotion across places and mind-sets. Cochlear implants can restore hearing for the hearing impaired or deaf but have been solely designed to restore speech intelligibility rather than other acoustic signals. For this reason, these devices fail at restoring music perception for the hearing impaired. According to the World Health Organization around 360 million people worldwide suffer from hearing loss. Hearing loss significantly limits the extent of interpersonal communication, often leads to social isolation, and has developed into a significant socio-economic factor. Over the last two decades research in cochlear implants has mainly been focused on improving speech performance in noise. However, recent scientific evidence points at music as an important auditory input for the development of the human brain – in terms of cognitive, emotional as well as auditory processing abilities. Music4u changes the fundamental perspective of cochlear implant research and therefore will use music technology as the key to improve the hearing performance and consequently the quality of life of cochlear implant users. Music4u investigates how to make instrumental music more accessible for cochlear implant users considering their individual hearing performance. First the preferred balance between the basic elements of the music to make it more enjoyable for cochlear implant users will be investigated. This fundamental understanding will be used to design a new signal processing algorithm dedicated to improve music perception and to accelerate their hearing performance. The algorithm is based on a deep neural network for source separation and posterior enhancement. The algorithm is intended to improve state of the art source separation algorithms for instrumental music. In a subsequent phase the complexity of the algorithm will be reduced to show its potential application in cochlear implant sound processor. The personalization of the algorithm is conducted through an electrophysiological measure of instrument discrimination. The first candidate for this measure is based on selective attention to polyphonic music excerpts with a low number of instruments. The performance with the electrophysiological measure will be compared to behavioral measures of instrument discrimination in actual cochlear implant users. In summary, the music4u project aims at conducting basic research to design a technology that can be personalized to each cochlear implant user to improve their music experience with the aim to improve their quality of life by integrating them into social-musical activities.

• DFG Programme: Research Grants