HomeCloudSoftwareHardware
Platform
  • RVC
  • Features
  • Depth
  • Deployment
  • Camera Sensors
  • Calibration
  • Environmental Specifications
  • Comparisons
RAE

API reference

package

rae_sdk

package
robot
package

rae_sdk.robot

package
api
module
audio
module
display
module
led
module
navigation
package
perception
module
robot
module
robot_options
module
state
package

rae_sdk.robot.api

module
openai
module
performance
package
ros
module

rae_sdk.robot.api.openai

class
OpenAIClient
class

rae_sdk.robot.api.openai.OpenAIClient

method
__init__(self, key = '', description = 'Assume you are a mobile robot that has a camera, speakers, microphone and LCD display. Let\'s say your main goal is to learn how to survive and interact with the world without causing harm to others.')
variable
property
method
generate_speech(self, text: str) -> str
Generate speech from text using OpenAI's TTS model.

Args:
----
    text (str): The text to generate speech from.

Returns
-------
    str: The path to the generated speech file
method
describe_image(self, image: np.ndarray, query = 'What\'s in this image?') -> str
Describe an image using OpenAI's Vision model.

Args:
----
    image (np.ndarray): The image to describe
    query (str): The query to ask the model. Default is "What's in this image?"

Returns
-------    
    str: The description of the image
method
respond(self, text: str) -> str
Generate a response from the robot using OpenAI's Chat model.

Args:
----
    text (str): The text to generate a response from.

Returns
-------
    str: The response from the robot
module

rae_sdk.robot.api.performance

function
function
variable
variable
variable
variable
function
measure_average_performance(func: Callable[..., Any]) -> Callable [ ... , Any ]
Report once every 5 minutes what the averages function duration is and what the max and min durations are.
variable
variable
variable
function
function
with_sql_exception_handling(func: Callable[..., Any]) -> Callable [ ... , Any ]
Try sql operation command, catch exception and finally close the cursor and the connection.
function
package

rae_sdk.robot.api.ros

module
ros_interface
module

rae_sdk.robot.api.ros.ros_interface

constant
class
ROSInterface
A class that manages ROS2 functionalities for a robot or a system.
class

rae_sdk.robot.api.ros.ros_interface.ROSInterface

method
__init__(self, robot_options: RobotOptions = RobotOptions())
Initialize the ROS2Manager instance.

Args:
----
    robot_options (RobotOptions): An object containing the robot's options.
property
method
start_hardware_process(self)
Start RAE hardware drivers in a separate process.
method
start(self)
Run RAE hardware drivers process.Initializes and starts the ROS2 node and executor. It sets up the ROS2 context and starts the ROS2 spin.

Args:
----
    start_hardware (bool): Whether to start the hardware process or not.
method
stop_ros_process(self)
Stop the ROS2 hardware process by terminating the related subprocess.
method
stop(self)
Shut down RAE drivers, ROS2 node and context.

This includes stopping the executor, destroying publishers subscribers, service clients, action clients, timers
and shutting down the ROS2 context.
method
method
method
method
method
method
method
method
method
method
method
get_frame_position(self, source_frame, target_frame) -> TransformStamped
Get the position of a frame relative to another frame.

Args:
----
    source_frame (str): The source frame.
    target_frame (str): The target frame.

Returns
-------
    TransformStamped: The position of the source frame relative to the target frame.
module

rae_sdk.robot.audio

class
AudioController
A class for controlling the robot's audio.
class

rae_sdk.robot.audio.AudioController

method
__init__(self, ros_interface)
method
method
save_recorded_sound(self, audio_data, output_file = '/app/mic_recording.wav')
Decode the Base64 audio data and save it as a WAV file.

Attributes
----------
    audio_data (str): Base64 encoded audio data.
    output_file (str, optional): Path to save the WAV file. Defaults to "/app/output.wav".
method
method
module

rae_sdk.robot.display

function
quaternion_to_rotation_matrix(q)
Convert a quaternion into a rotation matrix.

Arguments
-----------
q (tuple): A quaternion represented as (q_w, q_x, q_y, q_z).

Returns
-------
numpy.ndarray: A 3x3 rotation matrix.
class
DisplayController
A class for controlling the robot's display.
module

rae_sdk.robot.led

class
LEDController
A class for controlling the robot's LEDs.
class

rae_sdk.robot.led.LEDController

method
__init__(self, ros_interface)
method
hex_to_rgb(self, hex)
Convert a hex color to an RGB tuple.
method
normalize(self, num)
Normalize a number to a float between 0 and 1.
method
set_leds_from_payload(self, payload: dict)
Set the robot's LEDs to a given color.

Args:
----
    payload (dict): A dictionary containing the color to set the LEDs to. 
    Example payload struct: {'brightness': 50, 'color': '#FFFFFF', 'effect': 'pulse', 'interval': 5}
method
set_leds(self, color: str, brightness: int = 100, effect: str = 'solid', interval: int = 5)
Set the robot's LEDs to a given color.

Args:
----
    color (str): The color to set the LEDs to.
    brightness (int): The brightness of the LEDs. (Default: 100)
    effect (str): The effect to apply to the LEDs. (Default: "solid")
    interval (int): The interval of the effect. (Default: 5)
method
set_leds_from_msg(self, msg: LEDControl)
Set the robot's LEDs to a given color.

Args:
----
    msg (LEDControl): The message containing the color to set the LEDs to.
module

rae_sdk.robot.navigation

class
NavigationController
A class for controlling the robot's movement.
class

rae_sdk.robot.navigation.NavigationController

method
__init__(self, ros_interface)
method
move(self, linear, angular)
Move the robot in a given direction.

Args:
----
    linear (float): The linear velocity.
    angular (float): The angular velocity.
method
get_odom_position(self) -> TransformStamped
Get the robot's current position relative to odom frame. Returns None if the robot's position is not available.

Returns
-------
    TransformStamped: The robot's current position.
package

rae_sdk.robot.perception

module
detections
module
perception_system
module
pipeline
module

rae_sdk.robot.perception.detections

variable
class
Detections
function
class

rae_sdk.robot.perception.detections.Detections

variable
variable
module

rae_sdk.robot.perception.perception_system

variable
class
PerceptionSystem
A class for managing camera functionalities in a robot, interfacing with both depthai and robothub libraries.
class

rae_sdk.robot.perception.perception_system.PerceptionSystem

method
__init__(self, namespace = '')
Initialize the Camera instance.

Connect to the depthai device and initializes the ROS context manager.

Args:
----
    namespace (str, optional): The namespace for the ROS nodes. Defaults to ''.
method
__del__(self)
method
method
connect_to_device(self) -> bool
Connect to the depthai device and initialize the calibration handler.
method
start(self)
Start the ROS context manager and spins the ROS node.
method
stop(self)
Close the connection to the depthai device, ensuring a clean shutdown.
method
add_rh_stream(self, stream_name)
Add a video stream to RobotHub with the specified name.

Args:
----
    stream_name (str): The name of the stream to be added.
method
add_ros_img_stream(self, stream_name, topic_name, frame_name, socket, width = -1, height = -1, convertFromBitStream = False, frame_type = dai.RawImgFrame.Type.BGR888i)
Add a ROS video stream with the specified name and sets up the necessary configurations.

Args:
----
    stream_name (str): The name of the ROS stream to be added.
    topic_name (str): The name of the ROS topic to be published to.
    frame_name (str): The name of the ROS frame.
    socket (dai.CameraBoardSocket): The socket to be used for the stream.
    width (int, optional): The width of the stream. Defaults to -1.
    height (int, optional): The height of the stream. Defaults to -1.
    convertFromBitStream (bool, optional): Whether to convert the stream from a bitstream. Defaults to False.
    frame_type (dai.RawImgFrame.Type, optional): The type of the frame. Defaults to dai.RawImgFrame.Type.BGR888i.
method
add_ros_imu_stream(self, stream_name, topic_name, frame_name)
Add a ROS IMU stream with the specified name and sets up the necessary configurations.

Args:
----
    stream_name (str): The name of the ROS stream to be added.
    topic_name (str): The name of the ROS topic to be published to.
    frame_name (str): The name of the ROS frame.
method
add_ros_feature_stream(self, stream_name, topic_name, frame_name)
Add a ROS feature stream with the specified name and sets up the necessary configurations.

Args:
----
    stream_name (str): The name of the ROS stream to be added.
    topic_name (str): The name of the ROS topic to be published to.
    frame_name (str): The name of the ROS frame.
method
add_queue(self, name, callback = None)
Add a queue to the depthai device for processing callbacks.

Args:
----
    name (str): The name of the queue.
    callback (callable): The callback function to be added to the queue.
method
add_composable_node(self, package_name, plugin_name, options = dai_ros.ROSNodeOptions())
Add a composable node to the ROS context manager.

Args:
----
    package_name (str): The name of the ROS package.
    plugin_name (str): The name of the ROS plugin.
    options (dai_ros.ROSNodeOptions, optional): The options for the ROS node. Defaults to dai_ros.ROSNodeOptions().
method
start_pipeline(self, pipeline)
Start the camera pipeline, initializes the ROS node, and spins the ROS context manager.

Args:
----
    pipeline: The pipeline configuration for the camera.
variable
method
publish_rh(self, name, color_frame, timestamp, metadata)
Publish video data to a RobotHub stream.

Args:
----
    name (str): The name of the RobotHub stream.
    color_frame: The color frame data to be published.
    timestamp: The timestamp associated with the frame.
    metadata: Additional metadata for the frame.
method
publish_ros(self, name, msg)
Publish a message to a ROS topic.

Args:
----
    name (str): The name of the ROS topic.
    msg: The message to be published.
method
get_image(self, stream_name) -> np.ndarray
Retrieve an image from the specified stream.

Args:
----
    stream_name (str): The name of the stream to retrieve the image from.

Returns
-------
    Image: The image retrieved from the stream in OpenCV format.
method
method
method
variable
variable
variable
module

rae_sdk.robot.perception.pipeline

constant
function
function
function
module

rae_sdk.robot.robot

class
Robot
A class representing a robot, integrating various controllers for movement, display, and LED management and interfacing with ROS2 for communication and control.
class

rae_sdk.robot.robot.Robot

method
__init__(self, robot_options: RobotOptions = RobotOptions())
Initialize the Robot instance.

Args:
----
    robot_options (RobotOptions): An object containing the robot's options.
method
__del__(self)
method
stop(self)
Stop the ROS2 communications and deactivates the robot's controllers.

Ensures a clean shutdown of all components.
property
property
perception
Create perception system if it doesn't exist and return it.
property
property
property
property
property
module

rae_sdk.robot.robot_options

class
RobotOptions
A class for storing the robot's options.
module

rae_sdk.robot.state

class
StateInfo
A class for representing the robot's state.
class
StateController
A class for managing the robot's state.