# ObjectTracker

ObjectTracker associates detections across frames, assigning stable IDs and reporting per-track status. It takes detections from a
[DetectionNetwork](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/detection_network.md) or
[SpatialDetectionNetwork](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/spatial_detection_network.md) and
outputs [Tracklets](https://docs.luxonis.com/software-v3/depthai/depthai-components/messages/tracklets.md). When the upstream node
is a SpatialDetectionNetwork, each tracklet also carries spatialCoordinates (X/Y/Z in mm), and the tracker can estimate optional
3D velocity and speed.

Spatial data is available to tracklets automatically when detections include spatial coordinates. Spatially-aware association is a
separate tracker option and is disabled by default; enable it only when depth is reliable enough to potentially improve IDs in
crowded scenes, crossings, or partial occlusions.

In DepthAI v3, tracklets and frames additionally expose transformation metadata that allows remapping ROIs across streams (e.g.,
RGB → depth), as shown in the [Object Tracker
Remap](https://docs.luxonis.com/software-v3/depthai/examples/object_tracker/object_tracker_remap.md) example.

## How to place it

#### Python

```python
with dai.Pipeline() as pipeline:
    objectTracker = pipeline.create(dai.node.ObjectTracker)
```

#### C++

```cpp
dai::Pipeline pipeline;
auto objectTracker = pipeline.create<dai::node::ObjectTracker>();
```

## Inputs and Outputs

## Zero term tracking

Associates current-frame detections to existing tracks (no temporal prediction).

## Short term tracking

Propagates tracks between detector frames by extrapolating motion, which helps when the detector isn’t running every frame.

## Spatial data and association

Spatial data has two separate uses:

 * Tracklet output: when inputDetections comes from a SpatialDetectionNetwork, each Tracklet receives spatialCoordinates. If the
   tracker has a valid spatial state, it also outputs optional velocity and speed in m/s. RGB-only pipelines leave these optional
   fields empty, so check for None in Python or std::nullopt in C++.
 * Track association: call setSpatialAssociation(true) to use valid XYZ coordinates while matching detections to existing tracks.
   The spatial gate is applied alongside the 2D IoU gate, and spatial similarity is blended with the 2D association score.

Use spatial association when nearby objects overlap in image space but are separated in depth. Keep it disabled for RGB-only
tracking, or scenes where depth is too noisy to be a useful identity signal.

| Setter | Default | How to use it |
| --- | --- | --- |
| `setSpatialAssociation(bool)` | `false` | Enables spatially aware matching when valid spatial coordinates are present. |
| `setSpatialAssociationWeight(float)` | `0.5` | Blends 2D and spatial scores. Lower values favor image-space tracking, higher
values favor spatial consistency. This affects candidate scoring only; final acceptance still requires passing the 2D IoU gate. |
| `setSpatialDistanceThreshold(float)` | `1.5` | Base 3D gate in meters. Lower it for stricter matching or raise it for faster
motion or noisier depth. |
| `setSpatialDepthAwareScale(float)` | `0.35` | Grows the 3D gate with depth as `threshold * (1 + scale * depthMeters)`. Useful
for distant objects. |

#### Python

```python
objectTracker.setSpatialAssociation(True)
objectTracker.setSpatialAssociationWeight(0.5)
objectTracker.setSpatialDistanceThreshold(1.5)
objectTracker.setSpatialDepthAwareScale(0.35)
```

#### C++

```cpp
objectTracker->setSpatialAssociation(true);
objectTracker->setSpatialAssociationWeight(0.5f);
objectTracker->setSpatialDistanceThreshold(1.5f);
objectTracker->setSpatialDepthAwareScale(0.35f);
```

## Supported object tracker types

 * SHORT_TERM_IMAGELESS – tracks objects across frames using only bounding box position and size (no color data). Lightweight and
   fast, suitable for real-time use.

Select with setTrackerType(...). The ID policy is set with setTrackerIdAssignmentPolicy(...) and can be SMALLEST_ID or UNIQUE_ID.

> **Limit:**
> Up to
> **1000**
> tracked objects (cap via
> `setMaxObjectsToTrack(...)`
> )

## Examples of functionality

 * [Object Tracker](https://docs.luxonis.com/software-v3/depthai/examples/object_tracker/object_tracker.md)
 * [Object Tracker Replay](https://docs.luxonis.com/software-v3/depthai/examples/object_tracker/object_tracker_replay.md)
 * [Object Tracker Remap](https://docs.luxonis.com/software-v3/depthai/examples/object_tracker/object_tracker_remap.md)

## Reference

### dai::node::ObjectTracker

Kind: class

ObjectTracker node. Performs object tracking using Kalman filter and hungarian algorithm.

#### Input inputTrackerFrame

Kind: variable

Input ImgFrame message on which tracking will be performed. RGBp, BGRp, NV12, YUV420p types are supported. Default queue is
non-blocking with size 4.

#### Input inputDetectionFrame

Kind: variable

Input ImgFrame message on which object detection was performed. Default queue is non-blocking with size 4.

#### Input inputDetections

Kind: variable

Input message with image detection from neural network. Default queue is non-blocking with size 4.

#### Input inputConfig

Kind: variable

Input ObjectTrackerConfig message with ability to modify parameters at runtime. Default queue is non-blocking with size 4.

#### Output out

Kind: variable

Outputs Tracklets message that carries object tracking results.

#### Output passthroughTrackerFrame

Kind: variable

Passthrough ImgFrame message on which tracking was performed. Suitable for when input queue is set to non-blocking behavior.

#### Output passthroughDetectionFrame

Kind: variable

Passthrough ImgFrame message on which object detection was performed. Suitable for when input queue is set to non-blocking
behavior.

#### Output passthroughDetections

Kind: variable

Passthrough image detections message from neural network output. Suitable for when input queue is set to non-blocking behavior.

#### void setTrackerThreshold(float threshold)

Kind: function

Specify tracker threshold. parameters: threshold: Above this threshold the detected objects will be tracked. Default 0, all image
detections are tracked.

#### void setMaxObjectsToTrack(std::int32_t maxObjectsToTrack)

Kind: function

Specify maximum number of object to track. parameters: maxObjectsToTrack: Maximum number of object to track. Maximum 60 in case of
SHORT_TERM_KCF, otherwise 1000.

#### void setDetectionLabelsToTrack(std::vector< std::uint32_t > labels)

Kind: function

Specify detection labels to track. parameters: labels: Detection labels to track. Default every label is tracked from image
detection network output.

#### void setTrackerType(TrackerType type)

Kind: function

Specify tracker type algorithm. parameters: type: Tracker type.

#### void setTrackerIdAssignmentPolicy(TrackerIdAssignmentPolicy type)

Kind: function

Specify tracker ID assignment policy. parameters: type: Tracker ID assignment policy.

#### void setTrackingPerClass(bool trackingPerClass)

Kind: function

Whether tracker should take into consideration class label for tracking.

#### void setOcclusionRatioThreshold(float theshold)

Kind: function

Set the occlusion ratio threshold. Used to filter out overlapping tracklets. parameters: theshold: Occlusion ratio threshold.
Default 0.3.

#### void setTrackletMaxLifespan(uint32_t trackletMaxLifespan)

Kind: function

Set the tracklet lifespan in number of frames. Number of frames after which a LOST tracklet is removed. parameters:
trackletMaxLifespan: Tracklet lifespan in number of frames. Default 120.

#### void setTrackletBirthThreshold(uint32_t trackletBirthThreshold)

Kind: function

Set the tracklet birth threshold. Minimum consecutive tracked frames required to consider a tracklet as a new (TRACKED) instance.
parameters: trackletBirthThreshold: Tracklet birth threshold. Default 3.

#### void setSpatialAssociation(bool enabled)

Kind: function

Enable or disable spatially-aware association. If disabled, only 2D association is used. parameters: enabled:

#### void setSpatialAssociationWeight(float weight)

Kind: function

Set spatial association weight in [0,1]. parameters: weight: Spatial association weight in [0,1] used to blend 2D and spatial
association scores (0 = 2D-only scoring, 1 = spatial-only scoring). This weight affects candidate scoring only; final acceptance
still requires passing the 2D IoU threshold gate. Default is 0.5.

#### void setSpatialDistanceThreshold(float thresholdMeters)

Kind: function

Set base 3D gating threshold in meters for spatial association. parameters: thresholdMeters: Base spatial gating distance in
meters. Default is 1.5m.

#### void setSpatialDepthAwareScale(float scale)

Kind: function

Set depth-aware gating scale used for spatial association. Increases gating threshold with increased depth. parameters: scale:
Depth-aware gating scale factor. Default is 0.35

#### void setRunOnHost(bool runOnHost)

Kind: function

Specify whether to run on host or device By default, the node will run on device.

#### bool runOnHost()

Kind: function

Check if the node is set to run on host

#### void run()

Kind: function

#### DeviceNodeCRTP()

Kind: function

#### DeviceNodeCRTP(const std::shared_ptr< Device > & device)

Kind: function

#### DeviceNodeCRTP(std::unique_ptr< Properties > props)

Kind: function

#### DeviceNodeCRTP(std::unique_ptr< Properties > props, bool confMode)

Kind: function

#### DeviceNodeCRTP(const std::shared_ptr< Device > & device, std::unique_ptr< Properties > props, bool confMode)

Kind: function

### Need assistance?

Head over to [Discussion Forum](https://discuss.luxonis.com/) for technical support or any other questions you might have.