Interpretation of environments by incremental learning

• Applicant: Professor Dr. Jürgen Gall
• Subject Area: Geodesy, Photogrammetry, Remote Sensing, Geoinformatics, Cartography
• Term: from 2011 to 2020
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 166047863

Semantic information is essential to capture and identify inaccessible objects by an autonomous flying vehicle. For instance, it helps to fly with foresight und avoid collisions by identifying objects and anticipating their movements. It improves the 3D reconstruction of the environment by classifying surfaces that are difficult to reconstruct. First and foremost, however, it allows semantic queries by a user that steer the navigation and exploration behavior of a flying vehicle. For inspecting all windows of a building for instance, it is not necessary to reconstruct the entire building, which is time-consuming. Instead, it is sufficient to predict the locations of all observed and unobserved windows based on the current semantic interpretation of the environment. Project P7 aims to extract the necessary information from the incrementally reconstructed environments by machine learning approaches. Since the tasks ranging from semantic collision avoidance to semantic queries have different runtime and complexity requirements for the semantic representation, a multi-layer system will be developed. On the lowest layer image data is classified in real-time, on the highest layer the locations of class instances in an environment are probabilistically predicted, in particular, for regions that have not been explored yet. Since a key aspect of the second stage of the project is the incremental update of the map reconstructed from previous flights, P7 mainly focuses on the development of efficient incremental learning approaches. This is challenging since changes of time and lighting conditions also change the appearance of the training data. Furthermore, it will not be assumed that all classes are known a priori in order to solve complex tasks and to enhance the flexibility of semantic queries. Instead, the user can mark regions in the current map via an intuitive graphical user interface and declare the marked regions as instances of a new class. The goal of P7 is therefore not only the development of classifiers that can be efficiently updated as soon as new training samples of known classes are available, but that also learn efficiently new classes.

• DFG Programme: Research Units
• Subproject of: FOR 1505: Mapping on Demand
• Co-Investigator: Professor Dr. Lutz Plümer