robokudo.annotators.static_object_detector

Robokudo Static Object Detector Module

This module provides functionality for detecting objects at predefined locations using either manually configured bounding boxes or a world descriptor. The detector can create object hypotheses with poses, masks, and class labels.

Note

All poses are defined relative to the camera frame by default unless pose_in_world_coordinates is True.

Classes

StaticObjectMode

Generic enumeration.

StaticObjectDetectorAnnotator

Find a cluster based on a preconfigured Bounding Box, Pose and Class name.

Module Contents

class robokudo.annotators.static_object_detector.StaticObjectMode

Bases: enum.Enum

Generic enumeration.

Derive from this class to define new enumerations.

BOUNDING_BOX = 'bounding_box'
WORLD_DESCRIPTOR = 'world_descriptor'
class robokudo.annotators.static_object_detector.StaticObjectDetectorAnnotator(name: str = 'StaticObjectDetector', descriptor: StaticObjectDetectorAnnotator = Descriptor())

Bases: robokudo.annotators.core.BaseAnnotator

Find a cluster based on a preconfigured Bounding Box, Pose and Class name.

This annotator can:

  • Create object hypotheses at fixed locations using manual bounding box coordinates

  • Read predefined objects from the shared world to automatically infer bounding boxes

  • Generate pose annotations in either camera or world coordinates

  • Create masks for the detected regions

Note

The detector supports both Euler angles and quaternions for rotation specification. Parameters can be dynamically reconfigured through ROS dynamic reconfigure.

class Descriptor

Bases: robokudo.annotators.core.BaseAnnotator.Descriptor

Configuration descriptor for the StaticObjectDetectorAnnotator.

Defines all configurable parameters including:

  • Bounding box dimensions and position

  • Object knowledge database settings

  • Pose generation options

  • Coordinate frame settings

class Parameters

Parameters controlling the static object detection behavior.

bounding_box_x: int = 1
bounding_box_y: int = 1
bounding_box_width: int = 10
bounding_box_height: int = 10
mode: StaticObjectMode

Defines the mode which mainly decide which sources of information are used to generate the Object Hypothesis

class_name: str = 'unknown'

Define the class_name which is used for the object of interest Only used for StaticObjectMode.BOUNDING_BOX

class_names: typing_extensions.List[str] = []

Used for StaticObjectMode.WORLD_DESCRIPTOR

create_pose_annotation: bool = False

If True a Pose will be created for the object. Pose is relative to the camera frame by default.

Only effective in Mode=StaticObjectDetectorAnnotator.Mode.WORLD_DESCRIPTOR

create_bounding_box_annotation: bool = False

If True a BoundingBox will be created for the object.

Only effective in Mode=StaticObjectDetectorAnnotator.Mode.WORLD_DESCRIPTOR

create_mask: bool = True

If this is a true, a mask based on the ROI will be generated that marks every pixel as ON

parameters
color: typing_extensions.Optional[numpy.typing.NDArray] = None
depth: typing_extensions.Optional[numpy.typing.NDArray] = None
cloud: typing_extensions.Optional[open3d.geometry.PointCloud] = None
cam_intrinsics = None
object_body: typing_extensions.Optional[semantic_digital_twin.world_description.world_entity.Body] = None
object_bodies_by_name: typing_extensions.Dict[str, semantic_digital_twin.world_description.world_entity.Body]
detect_from_bb_descriptor(color_rgb: numpy.typing.NDArray) → robokudo.types.scene.ObjectHypothesis

Detect only based on the BB.

Parameters:

color_rgb – Image in RGB order

static _body_name(body: semantic_digital_twin.world_description.world_entity.Body) → str
static _select_body_shape_collection(body: semantic_digital_twin.world_description.world_entity.Body)
_get_body_bb_size_and_center(body: semantic_digital_twin.world_description.world_entity.Body) → tuple[numpy.ndarray, numpy.ndarray] | None
detect_from_body(body: semantic_digital_twin.world_description.world_entity.Body, object_id: int = 0, world_to_cam_transform_matrix: numpy.ndarray | None = None) → robokudo.types.scene.ObjectHypothesis | None

Detect from singular, passed SDT Body instance

detect_from_body_base(world_to_cam_transform_matrix: typing_extensions.Optional[numpy.typing.NDArray] = None) → typing_extensions.List[robokudo.types.scene.ObjectHypothesis]

Detect from a completed world descriptor

Returns:

A list of ObjectHypothesis objects

update() → py_trees.common.Status

Process current scene to detect configured static objects.

Steps:

  • Gets current color image, depth image and point cloud

  • If using a world descriptor, validates object class exists

  • Scales color image to match depth image

  • Creates object hypothesis with:

    • Bounding box from configuration or world descriptor

    • Pose annotation if enabled

    • Point cloud from ROI

    • Mask if enabled

Returns:

SUCCESS if detection completed, FAILURE if required transforms not found

Raises:

Exception – If camera parameters are invalid or missing

static add_classification_annotation(object_hypothesis: robokudo.types.scene.ObjectHypothesis, class_name: str) → None

Add a classification annotation to the given object hypothesis.

Parameters:

  • object_hypothesis – The object hypothesis to annotate.

  • class_name – The name of the annotation class.

get_rotation_list_based_on_parameters(rotation_x: float, rotation_y: float, rotation_z: float, rotation_w: float) → typing_extensions.List[float]

Return a quaternion based on the parametrization of the Annotator.

Parameters:

  • rotation_x – rotation about x-axis

  • rotation_y – rotation about y-axis

  • rotation_z – rotation about z-axis

  • rotation_w – rotation about w-axis

Returns:

4-dim list with Quaternion

get_cloud_from_2d_bb_roi(color_rgb: numpy.typing.NDArray) → open3d.geometry.PointCloud