Classification of the emotional impact initiated by film sequences Assessment of the emotional experience of a multimedia presentation consisting of video and audio.

• Applicants: Professor Dr.-Ing. Klaus Diepold; Professor Dr. Felix Schönbrodt
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing ; General, Cognitive and Mathematical Psychology ; Electronic Semiconductors, Components and Circuits, Integrated Systems, Sensor Technology, Theoretical Electrical Engineering ; Security and Dependability, Operating-, Communication- and Distributed Systems
• Term: from 2016 to 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 289289379

The concept Quality of Experience (QoE) tries to assess the subjective quality which an observer experiences when consuming multimedia content. In the current proposal, QoE should be extended by a new central component, namely the emotional state an observer experiences when watching a video. The core of our project is to extract emotionally relevant key features if videos using several machine learning approaches, which in turn are used to predict the experienced emotion of the observer.In order to realize this interdisciplinary research question technically, we draw upon and extend psychological models of emotion, which are then computationally implemented by engineers. Supervised machine learning approaches are trained by comparing the predicted with the actual emotional experience of an observer. This learning stage will use a crowdsourcing approach which allows to gather a large amount of data for relatively low costs. The large sample allows a broad generalizability of the results, and allows to investigate how personality characteristics of the observers modulate the emotional impact of the videos. The results from the online study will be cross-validated and extended in a controlled laboratory experiment. In this study, objective and indirect indicators of the experienced emotions are assessed (electromyography of facial expressions; codings of (micro-)emotions from video recordings of the observers). The interdisciplinary project is composed of researchers from psychology and computer engineers and works on the following research questions:a) Is it possible to determine a mapping of technical features onto emotionally relevant stimuli?b) Are psychological models able to correctly predict the experienced emotions based on relevant stimuli of the video and personality features of a person?Goals of the project are, on the one hand, to implement existing psychological theories of emotion, validate them using large data sets, and to develop them further based on these results. On the other hand, the technical framework should be employed to classify a large set of videos concerning the emotional impact on observers. Furthermore, the project will generate a standardized data base of videos, for which the emotional impact is very well known. We expect that such a data base will have a huge impact and benefit for future studies in the area of human emotions.

• DFG Programme: Research Grants