robokudo.annotators.image_cluster_extractor

Image-based object cluster extraction.

This module provides functionality for extracting object clusters from color images using HSV color segmentation. The main class ImageClusterExtractor implements color-based segmentation and contour detection to identify object clusters in RGB images.

Key features:

  • HSV color space thresholding

  • Contour detection and filtering

  • 3D point cloud generation from depth data

  • ROI and mask generation

  • Query-based color parameter adjustment

  • Visualization of detected clusters

Classes

ImageClusterExtractor

Extract object clusters from images using color segmentation.

Module Contents

class robokudo.annotators.image_cluster_extractor.ImageClusterExtractor(name: str = 'ImageClusterExtractor', descriptor: ImageClusterExtractor = Descriptor())

Bases: robokudo.annotators.core.BaseAnnotator

Extract object clusters from images using color segmentation.

This annotator performs the following steps:

  • Converts RGB image to HSV color space

  • Applies HSV thresholding based on configured parameters

  • Detects and filters contours based on size

  • Generates point clouds from depth data for each contour

  • Creates ObjectHypothesis annotations with ROIs and masks

  • Provides visualization of detected clusters

The HSV thresholds can be adjusted dynamically based on color queries.

class ViewMode

Visualization modes for the annotator output.

masked_object: int = 1

Show masked RGB image of detected objects

depth_mask: int = 2

Show depth mask of detected objects

class Descriptor

Bases: robokudo.annotators.core.BaseAnnotator.Descriptor

Configuration descriptor for ImageClusterExtractor.

Parameters:

  • HSV thresholding ranges

  • Contour filtering parameters

  • Point cloud generation settings

  • Color name to HSV range mappings

  • Outlier removal parameters

class Parameters

Parameter class containing all configurable settings.

hsv_min: typing_extensions.Tuple[int, int, int] = (150, 130, 85)
hsv_max: typing_extensions.Tuple[int, int, int] = (200, 255, 255)
erosion_iterations: int = 2
contour_min_size: int = 1000
color_name_to_hsv_range: typing_extensions.Dict[str, typing_extensions.Dict[str, typing_extensions.Tuple[int, int, int]]]
outlier_removal: bool = True
outlier_removal_nb_neighbors: int = 20
outlier_removal_std_ratio: float = 2.0
num_of_objects: int = 2
min_points_threshold: int = 62
parameters
color: typing_extensions.Optional[numpy.typing.NDArray] = None
depth: typing_extensions.Optional[numpy.typing.NDArray] = None
query = None
cam_intrinsics = None
display_mode = 1
adjust_hsv_threshold_to_query() → None

Adjust HSV thresholds based on color query.

Checks for a color query in the CAS and updates the HSV thresholding parameters if a matching color is found in the color_name_to_hsv_range mapping.

update() → py_trees.common.Status

Process input images to detect and annotate object clusters.

The method:

  • Scales color image to match depth image

  • Converts to HSV and applies thresholding

  • Detects and filters contours

  • Generates point clouds for each contour

  • Creates ObjectHypothesis annotations

  • Generates visualization output

Returns:

SUCCESS if clusters found, FAILURE if no clusters

Raises:

Exception – If no contours found or processing fails

key_callback(key: int) → None

Handle keyboard input to change visualization mode.

Parameters:

key – ASCII value of pressed key