OverviewHardwareSoftwareCloud
OAK Apps
DepthAI
AI
Software Stack
Troubleshooting

ON THIS PAGE

  • How to place it
  • Inputs and Outputs
  • Zero term tracking
  • Short term tracking
  • Spatial data and association
  • Supported object tracker types
  • Examples of functionality
  • Reference

ObjectTracker

Supported on:RVC2RVC4
ObjectTracker associates detections across frames, assigning stable IDs and reporting per-track status. It takes detections from a DetectionNetwork or SpatialDetectionNetwork and outputs Tracklets. When the upstream node is a SpatialDetectionNetwork, each tracklet also carries spatialCoordinates (X/Y/Z in mm), and the tracker can estimate optional 3D velocity and speed.Spatial data is available to tracklets automatically when detections include spatial coordinates. Spatially-aware association is a separate tracker option and is disabled by default; enable it only when depth is reliable enough to potentially improve IDs in crowded scenes, crossings, or partial occlusions.In DepthAI v3, tracklets and frames additionally expose transformation metadata that allows remapping ROIs across streams (e.g., RGB → depth), as shown in the Object Tracker Remap example.

How to place it

Python

Python
1with dai.Pipeline() as pipeline:
2    objectTracker = pipeline.create(dai.node.ObjectTracker)

C++

C++
1dai::Pipeline pipeline;
2auto objectTracker = pipeline.create<dai::node::ObjectTracker>();

Inputs and Outputs

Zero term tracking

Associates current-frame detections to existing tracks (no temporal prediction).

Short term tracking

Propagates tracks between detector frames by extrapolating motion, which helps when the detector isn’t running every frame.

Spatial data and association

Spatial data has two separate uses:
  • Tracklet output: when inputDetections comes from a SpatialDetectionNetwork, each Tracklet receives spatialCoordinates. If the tracker has a valid spatial state, it also outputs optional velocity and speed in m/s. RGB-only pipelines leave these optional fields empty, so check for None in Python or std::nullopt in C++.
  • Track association: call setSpatialAssociation(true) to use valid XYZ coordinates while matching detections to existing tracks. The spatial gate is applied alongside the 2D IoU gate, and spatial similarity is blended with the 2D association score.
Use spatial association when nearby objects overlap in image space but are separated in depth. Keep it disabled for RGB-only tracking, or scenes where depth is too noisy to be a useful identity signal.
SetterDefaultHow to use it
setSpatialAssociation(bool)falseEnables spatially aware matching when valid spatial coordinates are present.
setSpatialAssociationWeight(float)0.5Blends 2D and spatial scores. Lower values favor image-space tracking, higher values favor spatial consistency. This affects candidate scoring only; final acceptance still requires passing the 2D IoU gate.
setSpatialDistanceThreshold(float)1.5Base 3D gate in meters. Lower it for stricter matching or raise it for faster motion or noisier depth.
setSpatialDepthAwareScale(float)0.35Grows the 3D gate with depth as threshold * (1 + scale * depthMeters). Useful for distant objects.

Python

Python
1objectTracker.setSpatialAssociation(True)
2objectTracker.setSpatialAssociationWeight(0.5)
3objectTracker.setSpatialDistanceThreshold(1.5)
4objectTracker.setSpatialDepthAwareScale(0.35)

C++

C++
1objectTracker->setSpatialAssociation(true);
2objectTracker->setSpatialAssociationWeight(0.5f);
3objectTracker->setSpatialDistanceThreshold(1.5f);
4objectTracker->setSpatialDepthAwareScale(0.35f);

Supported object tracker types

  • SHORT_TERM_IMAGELESS – tracks objects across frames using only bounding box position and size (no color data). Lightweight and fast, suitable for real-time use.
Select with setTrackerType(...). The ID policy is set with setTrackerIdAssignmentPolicy(...) and can be SMALLEST_ID or UNIQUE_ID.

Examples of functionality

Reference

class

dai::node::ObjectTracker

#include ObjectTracker.hpp
variable
Input inputTrackerFrame
Input ImgFrame message on which tracking will be performed. RGBp, BGRp, NV12, YUV420p types are supported. Default queue is non-blocking with size 4.
variable
Input inputDetectionFrame
Input ImgFrame message on which object detection was performed. Default queue is non-blocking with size 4.
variable
Input inputDetections
Input message with image detection from neural network. Default queue is non-blocking with size 4.
variable
Input inputConfig
Input ObjectTrackerConfig message with ability to modify parameters at runtime. Default queue is non-blocking with size 4.
variable
Output out
Outputs Tracklets message that carries object tracking results.
variable
Output passthroughTrackerFrame
Passthrough ImgFrame message on which tracking was performed. Suitable for when input queue is set to non-blocking behavior.
variable
Output passthroughDetectionFrame
Passthrough ImgFrame message on which object detection was performed. Suitable for when input queue is set to non-blocking behavior.
variable
Output passthroughDetections
Passthrough image detections message from neural network output. Suitable for when input queue is set to non-blocking behavior.
function
void setTrackerThreshold(float threshold)
Specify tracker threshold.
Parameters
  • threshold: Above this threshold the detected objects will be tracked. Default 0, all image detections are tracked.
function
void setMaxObjectsToTrack(std::int32_t maxObjectsToTrack)
Specify maximum number of object to track.
Parameters
  • maxObjectsToTrack: Maximum number of object to track. Maximum 60 in case of SHORT_TERM_KCF, otherwise 1000.
function
void setDetectionLabelsToTrack(std::vector< std::uint32_t > labels)
Specify detection labels to track.
Parameters
  • labels: Detection labels to track. Default every label is tracked from image detection network output.
function
void setTrackerType(TrackerType type)
Specify tracker type algorithm.
Parameters
  • type: Tracker type.
function
void setTrackerIdAssignmentPolicy(TrackerIdAssignmentPolicy type)
Specify tracker ID assignment policy.
Parameters
  • type: Tracker ID assignment policy.
function
void setTrackingPerClass(bool trackingPerClass)
Whether tracker should take into consideration class label for tracking.
function
void setOcclusionRatioThreshold(float theshold)
Set the occlusion ratio threshold. Used to filter out overlapping tracklets.
Parameters
  • theshold: Occlusion ratio threshold. Default 0.3.
function
void setTrackletMaxLifespan(uint32_t trackletMaxLifespan)
Set the tracklet lifespan in number of frames. Number of frames after which a LOST tracklet is removed.
Parameters
  • trackletMaxLifespan: Tracklet lifespan in number of frames. Default 120.
function
void setTrackletBirthThreshold(uint32_t trackletBirthThreshold)
Set the tracklet birth threshold. Minimum consecutive tracked frames required to consider a tracklet as a new (TRACKED) instance.
Parameters
  • trackletBirthThreshold: Tracklet birth threshold. Default 3.
function
void setSpatialAssociation(bool enabled)
Enable or disable spatially-aware association. If disabled, only 2D association is used.
Parameters
  • enabled:
function
void setSpatialAssociationWeight(float weight)
Set spatial association weight in [0,1].
Parameters
  • weight: Spatial association weight in [0,1] used to blend 2D and spatial association scores (0 = 2D-only scoring, 1 = spatial-only scoring). This weight affects candidate scoring only; final acceptance still requires passing the 2D IoU threshold gate. Default is 0.5.
function
void setSpatialDistanceThreshold(float thresholdMeters)
Set base 3D gating threshold in meters for spatial association.
Parameters
  • thresholdMeters: Base spatial gating distance in meters. Default is 1.5m.
function
void setSpatialDepthAwareScale(float scale)
Set depth-aware gating scale used for spatial association. Increases gating threshold with increased depth.
Parameters
  • scale: Depth-aware gating scale factor. Default is 0.35
function
void setRunOnHost(bool runOnHost)
Specify whether to run on host or device By default, the node will run on device.
function
bool runOnHost()
Check if the node is set to run on host
function
void run()
inline function
DeviceNodeCRTP()
inline function
DeviceNodeCRTP(const std::shared_ptr< Device > & device)
inline function
DeviceNodeCRTP(std::unique_ptr< Properties > props)
inline function
DeviceNodeCRTP(std::unique_ptr< Properties > props, bool confMode)
inline function
DeviceNodeCRTP(const std::shared_ptr< Device > & device, std::unique_ptr< Properties > props, bool confMode)

Need assistance?

Head over to Discussion Forum for technical support or any other questions you might have.