robokudo.types.scene

Scene representation types for Robokudo.

This module provides types for representing and analyzing elements in a scene. It includes types for:

  • Analyzable scene elements

  • Object and human hypotheses

  • Region definitions

  • Part-whole relationships

  • Occupancy annotations

The types support 3D point clouds, image ROIs, and nested annotations.

Classes

AnalyzableAnnotation

An Annotation that describes an entity observed in the scene, that shall be analyzed further.

ObjectHypothesis

Hypothesis about an object in the scene.

HumanHypothesis

Hypothesis about a human in the scene.

RegionHypothesis

Hypothesis about a region in the scene.

ParthoodHypothesis

Base class for part-whole relationships in objects.

ParthoodComponentHypothesis

Base class for part-whole relationships in objects.

ParthoodFeatureHypothesis

Hypothesis about a feature part of an object.

OccupiedAnnotation

Annotation indicating region occupancy.

Module Contents

class robokudo.types.scene.AnalyzableAnnotation

Bases: robokudo.types.core.IdentifiableAnnotation

An Annotation that describes an entity observed in the scene, that shall be analyzed further. This is considered as a super class for everything that could potentially be necessary to look at. Examples for this are: Detected objects, Detected humans, but also subparts of detected objects. The latter includes specific features of an object or subcomponents that need their individual analysis.

Analyzables can get their own Annotations, might have 3D/6D points and their own ROI.

Variables:

  • annotations – List of associated annotations

  • points – Point cloud data

  • point_indices – Indices into point cloud

  • roi – Image region of interest

annotations: list = []
points: open3d.geometry.PointCloud = None
point_indices: list = []
roi: robokudo.types.cv.ImageROI
class robokudo.types.scene.ObjectHypothesis

Bases: AnalyzableAnnotation

Hypothesis about an object in the scene.

Basic object hypothesis without additional specialization. Inherits all capabilities from AnalyzableAnnotation.

class robokudo.types.scene.HumanHypothesis

Bases: ObjectHypothesis

Hypothesis about a human in the scene.

In contrast to ObjectHypothesis, we have many different aspects on a Object to annotate. Therefore, the semantics of self.points, self.centroid and self.roi on this Hypothesis are a bit different. These variables should always contain the maximum region of the person we have been able to detect so far. Examples:

  • When running only a face detection, they should be adjusted to the ROI for the face.

  • When running a full body detection + face detection, they should be adjusted to everything the full body detection and the face recognition was able to see. Basically a Union over the ROIs generated for both.

In general, PersonHypotheses should rely more on Annotations and also annotate the individual detected features, like body parts or activities.

Variables:

  • goal – Goal associated with the human

  • result – Result of human-related processing

goal = None
result = None
class robokudo.types.scene.RegionHypothesis

Bases: AnalyzableAnnotation, robokudo.types.core.Nameable

Hypothesis about a region in the scene.

A region in the scene. It doesn’t necessarily focus on one specific object, but is rather used as a data structure to analyze a specific part of the 3d space. Examples: A region above a sofa for free seat estimation, the space of a sink to check if it is filled with water, etc.

class robokudo.types.scene.ParthoodHypothesis

Bases: AnalyzableAnnotation, robokudo.types.core.Nameable

Base class for part-whole relationships in objects.

This annotation can be used to represent parts of an object. We mainly refer to ‘features’ and ‘components’. A logo at a specific place on an object would be a feature, while a control unit on an electrical device is a component.

Parthood Annotations are still a Hypothesis, because due to bad pose estimation we might actually not refer to the correct region of the parthood. Example: Pose estimation is off by 180 degrees around the up-axis and you are referring to a feature on the front face of an object.

class robokudo.types.scene.ParthoodComponentHypothesis

Bases: ParthoodHypothesis

Base class for part-whole relationships in objects.

This annotation can be used to represent parts of an object. We mainly refer to ‘features’ and ‘components’. A logo at a specific place on an object would be a feature, while a control unit on an electrical device is a component.

Parthood Annotations are still a Hypothesis, because due to bad pose estimation we might actually not refer to the correct region of the parthood. Example: Pose estimation is off by 180 degrees around the up-axis and you are referring to a feature on the front face of an object.

class robokudo.types.scene.ParthoodFeatureHypothesis

Bases: ParthoodHypothesis

Hypothesis about a feature part of an object.

Represents surface or visual features like:

  • Logos

  • Text

  • Decorative elements

  • Surface patterns

class robokudo.types.scene.OccupiedAnnotation

Bases: robokudo.types.core.Annotation

Annotation indicating region occupancy.

This annotation can be used to indicate that a region is occupied.

Variables:

occupied – Whether the region is occupied

occupied: bool = False