Project Details
Projekt
Development of a software system for automatic scene and person indexing in scientific video archives
Subject Area
Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
Modern and Contemporary History
Modern and Contemporary History
Term
from 2017 to 2021
Project identifier
Deutsche Forschungsgemeinschaft (DFG) - Project number 388420599
The German Broadcasting Archive (DRA) is a non-profit foundation under civil law, with offices in Frankfurt am Main and Potsdam-Babelsberg. The collection priorities of the archive at the Frankfurt location are audio recordings of contemporary history and music since the beginning of recording and historical recording media. In 1994, the DRA was extended to include the radio and television broadcasting archives of the former German Democratic Republic (GDR) initially at a location in Berlin, today in Potsdam-Babelsberg. In joint previous work, selected special GDR television broadcasts were digitized, and using innovative methods of content-based image and video analysis, have been made searchable. The material consists of approximately 3,000 hours of video footage, including the newscasts "Aktuelle Kamera", magazine broadcasts and 220 hours of the East German television film tradition. Through the use and development of automated methods for content-based video analysis, scientists have obtained new possibilities to carry out their searches for desired scenes, camera shots and persons, or for similar images. The sustainability of the achieved very good results of scene classification (detection of visual concepts) is intended to be further developed for future use. In the proposed project, a software system usable by archive staff will be developed to enable the DRA and other archives to easily integrate automatic video analysis methods for content-based image search. In this software system, deep learning methods will be employed, thereby making it possible at the same time to improve person and visual concept detection and expand them to other research-intensive parts of the television broadcasts. In particular, the project has the following objectives: 1. development of a sustainable software system for user-friendly expansion of two lexicons (concepts and persons) by archive staff, 2. integration of the software system in the digitalization workflows of the DRA to make automatic video analysis methods applicable to the total stock of television broadcasts in the archive, 3. improvement of the detection rates for concepts and persons by applying deep learning methods, 4. expansion of the visual concept lexicon by about 100 further concepts, 5. expansion of the person lexicon to about 100 persons of the GDR history, 6. improvement of the detection rates for concepts and persons through user feedback and similarity search, 7. development of appropriate visualizations for effective search. In this way, is it not only possible for scientists to carry out search queries on the basis of pre-defined concepts and persons, but they can also easily expand the extensive lexicons for visual concepts and persons for their own research tasks. The developed software tools will be made available to other scientific institutions as open source software.
DFG Programme
Research data and software (Scientific Library Services and Information Systems)
