Advancing Computational Musicology: Semi-supervised and unsupervised segmentation and annotation of musical collections (ACMus)

• Applicant: Professor Dr.-Ing. Karlheinz Brandenburg, since 12/2018
• Subject Area: Electronic Semiconductors, Components and Circuits, Integrated Systems, Sensor Technology, Theoretical Electrical Engineering ; Musicology
• Term: from 2018 to 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 403542342

The rapid advancement of ICT technologies in the last decades has translated into numerous research and development efforts for creation and management of digital cultural heritage resources. While the era of digital cultural heritage has resulted in wider accessibility of cultural content, it has also come with its own challenges, which include long-term access to information, sustainability, and rapid growth of data volume. Particularly for musical heritage, the main challenge lays in the need for semantic retrieval techniques capable of incorporating musical elements such us rhythm, harmony, melody, musical texture, and timbre. Such automatic retrieval techniques will open new paradigms for musicological research, enabling large-scale analysis of musical structure and similarity, rhythmic, melodic and harmonic pattern recognition, and analysis of time evolution of music traditions. With these techniques, new opportunities for enhancing existing musicological research will be provided, leading to increased efficiency, and to new possibilities for large-scale data manipulation and visualization not possible to date. The goal of this project is the development of semi-supervised and unsupervised semantic music retrieval methods for the automatic annotation of musical collections, focusing on four aspects: music, speech and singing voice discrimination, musical instrument ensemble recognition, musical meter recognition, and musical scale detection. The work program combines and adapts semi-supervised and unsupervised techniques for purposes of music structure annotation, acoustic scene segmentation, and environmental audio tagging, previous work on tuning and intonation analysis, musical instrument classification and parametrization, music/speech discrimination, and singing voice detection. The following research outcomes are expected from this project:• Efficient and reliable machine learning methods for automatic segmentation and classification of musical data – with respect to musical ensemble, meter, scale, and music/speech discrimination - that allow processing of large collections with minimal human intervention.• Novel and powerful workflows for musicological analysis based on the methodologies developed in this project and validated by expert musicologists.Due to the multidisciplinary nature of this research, the project team will be composed of researchers in the fields of signal processing, machine learning, music information retrieval MIR, musicology, and music. It will be conducted as a bilateral project between two German institutions, Fraunhofer Institute for Digital Media Technology IDMT and Technische Universität Ilmenau, and two Colombian universities, Universidad de Antioquia and Universidad Pontificia Bolivariana.

• DFG Programme: Research Grants
• Ehemalige Antragstellerin: Dr.-Ing. Estefanía Cano Cerón, until 12/2018