_images/rk_logo_v3-1000px.png

Robot Perception Framework#

_images/pr2_looking_at_table.jpg

RoboKudo is a perception framework targeted for robot manipulation tasks. It extends RoboSherlock and its key ideas by a flexible flow control based on behavior trees. It is a multi-expert approach to analyze unstructured sensor data and annotate particular regions of the data given the expertise of specific computer vision algorithms. In RoboKudo, perception tasks are formulated as pipelines, which are specialized variants of Sequences in behavior trees that allow for typical execution scenarios of perception approaches including access to a common analysis data structure (CAS) and attached visualizations for debugging. This concept allows us to combine the strength of behavior trees for flow control with integration and automatic selection for computer vision algorithms. RoboKudo is a knowledge-based perception approach, which allows it to be parameterized specifically for the perception tasks that a robot might encounter. The key idea is, that perception tasks can be formulated as a query for certain information and then the required computer vision algorithms are inferred and a suitable sequence of executing them is generated. This will also respect the dependencies that the computer vision algorithms might have. For example, the generation of point clouds from the sensor before executing any RGBD algorithm.