robokudo.io.semdt_raytracer_camera_interface

Simulated RGB-D camera interface backed by SemDT RayTracer.

Classes

SemDTRayTracerCameraInterface

Render RGB-D camera data from a Semantic Digital Twin world.

Module Contents

class robokudo.io.semdt_raytracer_camera_interface.SemDTRayTracerCameraInterface(camera_config)

Bases: robokudo.io.camera_interface.CameraInterface

Render RGB-D camera data from a Semantic Digital Twin world.

The interface loads a configured world descriptor, places a virtual camera in that world, and writes the rendered color image, depth image, segmentation, camera model, and ground-truth world reference into the CAS.

Note

The configured camera pose uses ROS optical-frame convention, while the SemDT ray tracer renders from a camera-link-like frame.

module_loader

Loader used to import configured SemDT world descriptors.

has_new_data() → bool

Report whether rendered camera data is available.

Returns:

Always True because simulated frames are rendered on demand.

set_data(cas: robokudo.cas.CAS) → None

Render a simulated RGB-D frame and write it into the CAS.

Parameters:

cas – CAS that receives rendered camera data and frame metadata.

_load_runtime_world() → semantic_digital_twin.world.World

Load the configured SemDT world and install it as runtime world.

Returns:

Runtime world instance used for rendering.

_ensure_world_frame(world: semantic_digital_twin.world.World) → semantic_digital_twin.world_description.world_entity.Body

Return the configured world frame body, creating it when needed.

Parameters:

world – Runtime world that contains the scene and camera frames.

Returns:

Body representing the configured world frame.

_ensure_camera_body(world: semantic_digital_twin.world.World, world_frame_body: semantic_digital_twin.world_description.world_entity.Body) → semantic_digital_twin.world_description.world_entity.Body

Return the configured camera body, creating it when needed.

Parameters:

• world – Runtime world that contains the scene and camera frames.

• world_frame_body – Parent frame for a newly created camera body.

Returns:

Body representing the configured camera frame.

_set_camera_pose(world: semantic_digital_twin.world.World, world_frame_body: semantic_digital_twin.world_description.world_entity.Body, camera_body: semantic_digital_twin.world_description.world_entity.Body) → Tuple[semantic_digital_twin.spatial_types.HomogeneousTransformationMatrix, semantic_digital_twin.spatial_types.HomogeneousTransformationMatrix]

Apply the configured camera pose to the runtime world.

Parameters:

• world – Runtime world that owns the camera body connection.

• world_frame_body – Reference frame for the configured camera pose.

• camera_body – Camera frame body that receives the configured pose.

Returns:

RayTracer render pose and ROS optical camera pose.

static _camera_link_to_optical_np() → numpy.ndarray

Return the transform from camera-link frame to ROS optical frame.

Returns:

Numpy homogeneous transformation matrix.

static _camera_optical_to_link_np() → numpy.ndarray

Return the transform from ROS optical frame to camera-link frame.

Returns:

Numpy homogeneous transformation matrix.

static _render_segmentation_and_depth(ray_tracer: semantic_digital_twin.spatial_computations.raytracer.RayTracer, camera_to_world: semantic_digital_twin.spatial_types.HomogeneousTransformationMatrix, resolution: int, fov_deg: float, min_distance: float, max_distance: float) → Tuple[numpy.ndarray, numpy.ndarray]

Render object segmentation and projective depth.

Parameters:

• ray_tracer – Renderer that creates camera rays and returns their scene intersections.

• camera_to_world – Camera pose used by the RayTracer renderer.

• resolution – Square image resolution.

• fov_deg – Horizontal camera field of view.

• min_distance – Minimum valid ray-hit distance.

• max_distance – Maximum valid ray-hit distance.

Returns:

Segmentation indices and depth image in meters.

_render_color_image(world: semantic_digital_twin.world.World, ray_tracer, camera_to_world: semantic_digital_twin.spatial_types.HomogeneousTransformationMatrix, segmentation: numpy.ndarray, resolution: int, fov_deg: float) → Tuple[numpy.ndarray, Dict[str, str]]

Render a BGR color image for the current frame.

Parameters:

• world – Runtime world that provides semantic body colors.

• ray_tracer – SemDT ray tracer used for optional mesh rendering.

• camera_to_world – Camera pose used by the RayTracer renderer.

• segmentation – Body-index segmentation image.

• resolution – Square image resolution.

• fov_deg – Horizontal camera field of view.

Returns:

BGR image and optional RGB-to-object-name color map.

_try_render_trimesh_rgb(ray_tracer, camera_to_world: semantic_digital_twin.spatial_types.HomogeneousTransformationMatrix, resolution: int, fov_deg: float) → numpy.ndarray | None

Render textured mesh colors through the RayTracer scene.

Parameters:

• ray_tracer – SemDT ray tracer that owns the Trimesh scene.

• camera_to_world – Camera pose used by the RayTracer renderer.

• resolution – Square image resolution.

• fov_deg – Horizontal camera field of view.

Returns:

BGR image when rendering succeeds, otherwise None.

_render_semantic_rgb(world: semantic_digital_twin.world.World, segmentation: numpy.ndarray) → Tuple[numpy.ndarray, Dict[str, str]]

Render deterministic semantic RGB colors from segmentation labels.

Parameters:

• world – Runtime world that maps body indices to bodies.

• segmentation – Body-index segmentation image.

Returns:

RGB image and RGB-to-object-name color map.

static _rgb_for_body(body: semantic_digital_twin.world_description.world_entity.Body) → numpy.ndarray

Return the semantic RGB color for a world body.

Parameters:

body – World body whose collision or visual color is used.

Returns:

RGB color encoded as three unsigned bytes.

static _depth_m_to_mm(depth_m: numpy.ndarray) → numpy.ndarray

Convert meter depth values to unsigned millimeter depth values.

Parameters:

depth_m – Depth image in meters with negative values for misses.

Returns:

Depth image in millimeters with misses encoded as zero.

static _build_camera_models(frame_id: str, resolution: int, fov_deg: float) → Tuple[sensor_msgs.msg.CameraInfo, open3d.camera.PinholeCameraIntrinsic]

Build ROS and Open3D pinhole camera models.

Parameters:

• frame_id – Camera frame name stored in the ROS camera info header.

• resolution – Square image resolution.

• fov_deg – Horizontal camera field of view.

Returns:

ROS camera info and matching Open3D intrinsic model.