RoboSherlock - Scaling Robot Perception towards Everyday Manipulation through Unstructured Information Processing

• Applicant: Professor Michael Beetz, Ph.D.
• Subject Area: Automation, Mechatronics, Control Systems, Intelligent Technical Systems, Robotics ; Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2015 to 2018
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 260410154

We propose the design, implementation, and empirical analysis of RoboSherlock, a novel robot perception framework for facilitating everyday manipulation activities with objects of daily use in human living environments. The aim of the RoboSherlock project is to equip robots with perception systems that are flexible, accurate, and robust without having to interrupt ongoing activity. This is achieved by equipping the perception system with the following capabilities:- lifelong learning and adaptation for everyday activity,- powerful first-order probabilistic reasoning,- knowledge- and model-based perception methods, and- proactive and anticipatory perception.In order to implement RoboSherlock we consider perception as an unstructured information processing and management problem. Unstructured information management (UIM) has proven itself to be a powerful paradigm for scaling intelligent information and question answering systems towards real-world complexity. Complexity in UIM is handled by identifying (or hypothesizing) pieces of structured information in unstructured documents, by applying ensembles of experts for annotating information pieces, and by testing and integrating these isolated annotations into a comprehensive interpretation of the document. In the RoboSherlock project we will transfer these methods to robot perception in order to scale it to realistic perception tasks in realistic environments.The perception capabilities implemented in RoboSherlock will be evaluated and empirically analyzed in the context of a robotic agent that performs fetch and place tasks for everyday activities, 5 days a week for several weeks, such as setting the table, cleaning up, loading and unloading the dishwasher, etc.

• DFG Programme: Research Grants