OverviewHardwareSoftwareCloud
OAK Apps
DepthAI
AI
Software Stack
Troubleshooting

ON THIS PAGE

  • Usage
  • Related examples
  • Reference

ImgTransformations

ImgTransformations are not a standalone message type, but are metadata attached to several DepthAI messages that describe all the parameters of the virtual camera that would produce such an image. The stored information includes the full history of transformations (crop, resize, rotate, align, warp), distortion coefficients, image dimensions and extrinsics to a common reference coordinate system. They are tracked explicitly across all nodes, instead of forcing you to manually reconstruct them from the node configuration. Nodes can use ImgTransformations for a variety of purposes, including:
  • Aligning RGB, depth, and other derived streams when geometry needs to move between them.
  • Remapping rectangles, points, and other image-space geometry after an ImageManip, ImageAlign, or similar operation.
  • Reading intrinsics, extrinsics or any other camera properties for the image that was actually produced, rather than assuming the original sensor intrinsics still apply.
Python
1frame = queue.get()
2transform = frame.getTransformation()
3intrinsics = transform.getSourceIntrinsicMatrix()

Usage

The following messages have an ImgTransformations field:

Related examples

Reference

struct

dai::ImgTransformation

#include ImgTransformations.hpp
function
ImgTransformation()
inline function
ImgTransformation(size_t width, size_t height)
inline function
ImgTransformation(size_t srcWidth, size_t srcHeight, size_t width, size_t height)
inline function
ImgTransformation(size_t width, size_t height, std::array< std::array< float, 3 >, 3 > sourceIntrinsicMatrix)
inline function
ImgTransformation(size_t width, size_t height, std::array< std::array< float, 3 >, 3 > sourceIntrinsicMatrix, CameraModel distortionModel, std::vector< float > distortionCoefficients)
inline function
ImgTransformation(size_t width, size_t height, std::array< std::array< float, 3 >, 3 > sourceIntrinsicMatrix, CameraModel distortionModel, std::vector< float > distortionCoefficients, Extrinsics extrinsics)
function
dai::Point2f transformPoint(dai::Point2f point)
Transform a point from the source frame to the current frame.
Parameters
  • point: Point to transform
Returns
Transformed point
function
dai::RotatedRect transformRect(dai::RotatedRect rect)
Transform a rotated rect from the source frame to the current frame.
Parameters
  • rect: Rectangle to transform
Returns
Transformed rectangle
function
dai::Point2f invTransformPoint(dai::Point2f point)
Transform a point from the current frame to the source frame.
Parameters
  • point: Point to transform
Returns
Transformed point
function
dai::RotatedRect invTransformRect(dai::RotatedRect rect)
Transform a rotated rect from the current frame to the source frame.
Parameters
  • rect: Rectangle to transform
Returns
Transformed rectangle
function
std::pair< size_t, size_t > getSize()
Retrieve the size of the frame. Should be equal to the size of the corresponding ImgFrame message.
Returns
Size of the frame
function
std::pair< size_t, size_t > getSourceSize()
Retrieve the size of the source frame from which this frame was derived.
Returns
Size of the frame
function
std::array< std::array< float, 3 >, 3 > getMatrix()
Retrieve the transformation matrix from the source frame to the current frame.
Returns
Transformation matrix
function
std::array< std::array< float, 3 >, 3 > getMatrixInv()
Retrieve the inverse transformation matrix from the current frame to the source frame.
Returns
Inverse transformation matrix
function
std::array< std::array< float, 3 >, 3 > getSourceIntrinsicMatrix()
Retrieve the intrinsic matrix of the source sensor.
Returns
Intrinsic matrix
function
std::array< std::array< float, 3 >, 3 > getSourceIntrinsicMatrixInv()
Retrieve the inverse intrinsic matrix of the source sensor.
Returns
Inverse intrinsic matrix
function
CameraModel getDistortionModel()
Retrieve the distortion model of the source sensor
Returns
Distortion model
function
std::vector< float > getDistortionCoefficients()
Retrieve the distortion coefficients of the source sensor
Returns
vector of distortion coefficients
function
Extrinsics getExtrinsics()
Retrieve the extrinsics to the source sensor.
Returns
Extrinsics
function
bool isEqualTransformation(const ImgTransformation & other)
Two transformations are equal if the transformation matrices, intrinsic matrices, distortion models, distortion coefficients, extrinsics, and sizes are all equal.
Parameters
  • other: Transformation to compare with
Returns
True if the transformations are equal, false otherwise
inline function
std::array< std::array< float, 3 >, 3 > getTransformationMatrix()
inline function
std::array< std::array< float, 3 >, 3 > getTransformationMatrixInv()
function
std::array< std::array< float, 3 >, 3 > getIntrinsicMatrix()
Retrieve the total intrinsic matrix calculated from transform * intrinsic.
Returns
total intrinsic matrix
function
std::array< std::array< float, 3 >, 3 > getIntrinsicMatrixInv()
Retrieve the inverse of the total intrinsic matrix.
Returns
inverse total intrinsic matrix
function
float getDFov(bool source)
Retrieve the diagonal field of view of the image.
Parameters
  • source: If true, the source field of view will be returned. Otherwise, the current field of view will be returned.
Returns
Diagonal field of view in degrees
function
float getHFov(bool source)
Retrieve the horizontal field of view of the image.
Parameters
  • source: If true, the source field of view will be returned. Otherwise, the current field of view will be returned.
Returns
Horizontal field of view in degrees
function
float getVFov(bool source)
Retrieve the vertical field of view of the image.
Parameters
  • source: If true, the source field of view will be returned. Otherwise, the current field of view will be returned.
Returns
Vertical field of view in degrees
function
std::vector< dai::RotatedRect > getSrcCrops()
function
bool getSrcMaskPt(size_t x, size_t y)
Returns true if the point is inside the transformed region of interest (determined by crops used).
function
bool getDstMaskPt(size_t x, size_t y)
Returns true if the point is inside the image region (not in the background region).
function
ImgTransformation & addTransformation(std::array< std::array< float, 3 >, 3 > matrix)
Add a new transformation.
Parameters
  • matrix: Transformation matrix
function
ImgTransformation & addCrop(int x, int y, int width, int height)
Add a crop transformation.
Parameters
  • x: X coordinate of the top-left corner of the crop
  • y: Y coordinate of the top-left corner of the crop
  • width: Width of the crop
  • height: Height of the crop
function
ImgTransformation & addPadding(int top, int bottom, int left, int right)
Add a pad transformation. Works like crop, but in reverse.
Parameters
  • top: Padding on the top
  • bottom: Padding on the bottom
  • left: Padding on the left
  • right: Padding on the right
function
ImgTransformation & addFlipVertical()
Add a vertical flip transformation.
function
ImgTransformation & addFlipHorizontal()
Add a horizontal flip transformation.
function
ImgTransformation & addRotation(float angle, dai::Point2f rotationPoint)
Add a rotation transformation.
Parameters
  • angle: Angle in degrees
  • rotationPoint: Point around which to rotate
function
ImgTransformation & addScale(float scaleX, float scaleY)
Add a scale transformation.
Parameters
  • scaleX: Scale factor in the horizontal direction
  • scaleY: Scale factor in the vertical direction
function
ImgTransformation & addSrcCrops(const std::vector< dai::RotatedRect > & crops)
function
ImgTransformation & setSize(size_t width, size_t height)
function
ImgTransformation & setSourceSize(size_t width, size_t height)
function
ImgTransformation & setExtrinsics(const Extrinsics & extrinsics)
function
ImgTransformation & setIntrinsicMatrix(std::array< std::array< float, 3 >, 3 > intrinsicMatrix)
function
ImgTransformation & setDistortionModel(CameraModel model)
function
ImgTransformation & setDistortionCoefficients(std::vector< float > coefficients)
function
dai::Point2f remapPointTo(const ImgTransformation & to, dai::Point2f point)
Remap a point from this transformation to another. If the intrinsics are different (e.g. different camera), the function will also use the intrinsics to remap the point.
Parameters
  • to: Transformation to remap to
  • point: Point to remap
Parameters
This function assumes both transformations have the same source (eg. same source camera socket). If they don't, remapping will be inaccurate.
function
dai::Point2f remapPointFrom(const ImgTransformation & from, dai::Point2f point)
Remap a point to this transformation from another. If the intrinsics are different (e.g. different camera), the function will also use the intrinsics to remap the point.
Parameters
  • from: Transformation to remap from
  • point: Point to remap
function
dai::RotatedRect remapRectTo(const ImgTransformation & to, dai::RotatedRect rect)
Remap a rotated rect from this transformation to another. If the intrinsics are different (e.g. different camera), the function will also use the intrinsics to remap the rect.
Parameters
  • to: Transformation to remap to
  • rect: RotatedRect to remap
function
dai::RotatedRect remapRectFrom(const ImgTransformation & from, dai::RotatedRect rect)
Remap a rotated rect to this transformation from another. If the intrinsics are different (e.g. different camera), the function will also use the intrinsics to remap the rect.
Parameters
  • from: Transformation to remap from
  • rect: RotatedRect to remap
function
dai::Point2f project3DPoint(const dai::Point3f & point)
Project a 3D spatial point into 2D point in the current frame defined by this transformation.
Parameters
  • point: 3D point to project
Returns
Projected 2D point in the current frame
Parameters
This function assumes that the point is in the coordinate system of the current frame.
function
dai::Point2f projectPointTo(const ImgTransformation & to, dai::Point2f & point, float depth)
Project a 2D point from the source frame defined by this transformation into a 2D point in the target frame defined by the to transformation. This function will use the depth of the point to project it into 3D space and then reproject it back to 2D in the target frame.
Parameters
  • to: Target transformation to project to
  • point2f: Source 2D point in the current frame
  • depth: (mm) Depth of the point to project
Returns
Projected 2D point in the target frame (to transformation)
function
dai::RotatedRect projectRectTo(const ImgTransformation & to, RotatedRect & rect, float depth)
Project a rotated rectangle from the source frame defined by this transformation into the target frame defined by the to transformation. This function assumes the rectangle is parallel to the image plane (i.e. the entire rectangle is at the same depth away). The function fits the smallest possible rectangle on to the projected corners of the original rectangle.
Parameters
  • to: Target transformation to project to
  • rect: Source rotated rectangle in the current frame
  • depth: (mm) Depth of the rectangle to project
function
dai::Point2f project3DPointTo(const ImgTransformation & to, const dai::Point3f & point)
Project a 3D spatial point from the source coordinate system (this transformation) into a 2D point in the target frame (to transformation).
Parameters
  • to: Target transformation to project to
  • point3f: 3D point to project
Returns
Projected 2D point in the target frame (to transformation)
Parameters
This function assumes that the point3f is in the coordinate system of the current frame.
function
dai::Point2f project3DPointFrom(const ImgTransformation & from, const dai::Point3f & point)
Project a 3D point from the source frame (from transformation) into a 2D point in the current frame (this transformation).
Parameters
  • from: Transformation to project from
  • point3f: 3D point to project
Returns
Projected 2D point in the current frame
Parameters
This function assumes that the point3f is in the coordinate system of the source frame.
function
dai::Point3f remap3DPointTo(const ImgTransformation & to, const dai::Point3f & point)
Remap a 3D point from the source coordinate system of this transformation to the coordinate system of the target transformation (to transformation).
Parameters
  • to: Target transformation to remap to
  • point: 3D point to remap
Returns
Remapped 3D point in the target coordinate system
Parameters
This function assumes that the point is in the coordinate system of the current frame.
function
dai::Point3f remap3DPointFrom(const ImgTransformation & from, const dai::Point3f & point)
Remap a 3D point to the coordinate system of this transformation from the source coordinate system of the from transformation.
Parameters
  • from: Transformation to remap from
  • point: 3D point to remap
Returns
Remapped 3D point in the current coordinate system
Parameters
This function assumes that the point is in the coordinate system of the source frame.
function
std::array< std::array< float, 4 >, 4 > getExtrinsicsTransformationMatrixTo(const ImgTransformation & to, bool useSpecTranslation, LengthUnit sourceUnit)
Get the extrinsic transformation matrix from the source coordinate system of this transformation to the target coordinate system of the to transformation.
Parameters
  • to: Target transformation to get extrinsics to
  • useSpecTranslation: If true, the translation vector w.r.t. the CAD design will be used instead of the translation vector obtained through calibration.
  • sourceUnit: The desired measurement unit in which to return the transformation matrix in.
Returns
4x4 homogeneous transformation matrix representing the extrinsics from this transformation to the target transformation
Parameters
Both transformations must have a common toCameraSocket. Otherwise extrinsics cannot be calculated.
function
std::array< std::array< float, 3 >, 3 > getRotationMatrixTo(const ImgTransformation & to)
Get the extrinsic rotation matrix from the source coordinate system of this transformation to the target coordinate system of the to transformation.
Parameters
  • to: Transformation to get extrinsics to
Returns
3x3 rotation matrix representing the extrinsic rotation from this transformation to the target transformation
function
std::array< float, 3 > getTranslationVectorTo(const ImgTransformation & to, bool useSpecTranslation, LengthUnit sourceUnit)
Get the extrinsic translation vector from the source coordinate system of this transformation to the target coordinate system of the to transformation.
Parameters
  • to: Transformation to get extrinsics to
Returns
3x1 translation vector representing the extrinsic translation from this transformation to the target transformation
function
bool isAlignedTo(const ImgTransformation & to)
Check if the transformations are aligned
Parameters
  • to: Transformation to compare with
function
bool isValid()
Check if the transformations are valid. The transformations are valid if the source frame size and the current frame size are set.
function
DEPTHAI_SERIALIZE(ImgTransformation, extrinsics, transformationMatrix, transformationMatrixInv, sourceIntrinsicMatrix, sourceIntrinsicMatrixInv, distortionModel, distortionCoefficients, srcWidth, srcHeight, width, height, srcCrops)

Need assistance?

Head over to Discussion Forum for technical support or any other questions you might have.