RoboSherlock - Scaling Robot Perception towards Everyday Manipulation through Unstructured Information Processing
Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
Final Report Abstract
In RoboSherlock we proposed research in robot perception for autonomous manipulation tasks in human living environments that involve typical objects of daily use. Our aim was to investigate pervasively operating perception systems that enable robotic agents to fluently perform everyday manipulation tasks. As future outlook we have identified several interesting problems that would need addressing: • Pervasive learning: The RoboSherlock perception task language enables us to investigate how the perceptual capabilities of a robotic system behave when instructed to perform different tasks. Together with the logging og perceptual memories this can serve as a starting point for learning generalized perceptual capabilities. • Perception for manipulation: In the current project our attention was mostly focused on static scenes, where changes happen at a slower pace (e.g. pick and place). A natural extension to this would be perception for manipulation such as perception of action effects or visual servoing capabilities. • Sustainable development: Lastly the RoboSherlock framework continues to be an integral part of the robotic system built at that Institute for Artificial Intelligence. As such it is necessary to push RoboSherlock towards becoming an industrial strength open source perception framework. In conclusion, we consider the project to be a huge success. The perception system developed is actively used by members of the research institute and students learning to program robots. The framework allows researchers to focus on questions regarding perception that have not been possible before, e.g.: the interaction of perception with other robotic system components, and how these influence behavior, or teaching robots through virtual realities to ask the right perception query at the right time. These are only a few examples, but they show how RoboSherlock enables us to think about perception problems in robotics on a larger scale, and see them as a part of a whole.
Publications
- RoboSherlock: Unstructured information processing for robot perception. IEEE International Conference on Robotics and Automation (2015). Best Service Robotics paper
Michael Beetz, Ferenc Balint-Benczedi, Nico Blodow, Daniel Nyga, Thiemo Wiedemeyer, Zoltan-Marton Csaba
(See online at https://doi.org/10.1109/ICRA.2015.7139395) - Storing and Retrieving Perceptual Episodic Memories for Long-term Manipulation Tasks, IEEE International Conference on Advanced Robotics (ICAR) 2017
Ferenc Balint-Benczedi, Zoltan-Csaba Marton, Maximilian Durner, Michael Beetz
(See online at https://doi.org/10.1109/ICAR.2017.8023492)
