HomeCloudSoftwareHardware
DepthAI Components
DepthAI Tutorials
DepthAI Perception
Tools
ROS
Neural Networks
DepthAI API References

ON THIS PAGE

  • ObjectTracker
  • How to place it
  • Inputs and Outputs
  • Zero term tracking
  • Short term tracking
  • Supported object tracker types
  • Maximum number of tracked objects
  • Usage
  • Examples of functionality
  • Reference

ObjectTracker

Object tracker tracks detected objects from the ImgDetections using Kalman filter and hungarian algorithm.

How to place it

Python
C++
Python
1pipeline = dai.Pipeline()
2objectTracker = pipeline.create(dai.node.ObjectTracker)

Inputs and Outputs

Command Line
1/
2                      ┌───────────────────┐
3  inputDetectionFrame │                   │passthroughDetectionFrame
4  ───────────────────►│-------------------├─────────────────────────►
5                      │                   │                      out
6                      │      Object       ├─────────────────────────►
7  inputTrackerFrame   │      Tracker      │  passthroughTrackerFrame
8  ───────────────────►│-------------------├─────────────────────────►
9  inputDetections     │                   │    passthroughDetections
10  ───────────────────►│-------------------├─────────────────────────►
11                      └───────────────────┘
Message types

Zero term tracking

Zero term tracking performs object association, which means that it does not conduct prediction and tracking based on previous tracking history. Object association would mean that detected objects from an external detector are mapped with tracked objects which has been detected and is being tracked from previous frames.

Short term tracking

Short-term tracking allows to track objects between frames, thereby reducing the need to run object detection on each frame. This works great with NN models that can't achieve 30FPS (eg. YoloV5); tracker can provide tracklets when there was no inference, so the whole system can run at 30FPS.

Supported object tracker types

  • SHORT_TERM_KCF: Kernelized Correlation Filter tracking. KCF utilizes properties of circulant matrix to enhance the processing speed. Paper here.
  • SHORT_TERM_IMAGELESS: Short-term tracking allows to track objects on frames where object detection was skipped, by extrapolating object trajectory from previous detections.
  • ZERO_TERM_COLOR_HISTOGRAM: Utilizes position, shape and input image information such as RGB histogram to perform object tracking.
  • ZERO_TERM_IMAGELESS: Only utilizes rectangular shape of detected object and position information for object tracking. It does not use color information of tracking objects. It achieves higher throughput than ZERO_TERM_COLOR_HISTOGRAM. User needs to consider the trade-off between throughput and accuracy when choosing the object tracker type.
A similar comparison of object trackers with more information can be found here.

Maximum number of tracked objects

ObjectTracker node can track up to 60 objects at once. At the moment the firmware crashes if there are more than 60 objects to track.

Usage

Python
C++
Python
1pipeline = dai.Pipeline()
2objectTracker = pipeline.create(dai.node.ObjectTracker)
3
4objectTracker.setDetectionLabelsToTrack([15])  # Track only person
5# Possible tracking types: ZERO_TERM_COLOR_HISTOGRAM, ZERO_TERM_IMAGELESS, SHORT_TERM_IMAGELESS, SHORT_TERM_KCF
6objectTracker.setTrackerType(dai.TrackerType.ZERO_TERM_COLOR_HISTOGRAM)
7# Take the smallest ID when new object is tracked, possible options: SMALLEST_ID, UNIQUE_ID
8objectTracker.setTrackerIdAssignmentPolicy(dai.TrackerIdAssignmentPolicy.SMALLEST_ID)
9
10# You have to use Object tracker in combination with detection network
11# and an image frame source - mono/color camera or xlinkIn node

Examples of functionality

Reference

class

depthai.node.ObjectTracker(depthai.Node)

method
setDetectionLabelsToTrack(self, labels: list [ int ])
Specify detection labels to track.

Parameter ``labels``:
    Detection labels to track. Default every label is tracked from image
    detection network output.
method
setMaxObjectsToTrack(self, maxObjectsToTrack: int)
Specify maximum number of object to track.

Parameter ``maxObjectsToTrack``:
    Maximum number of object to track. Maximum 60 in case of SHORT_TERM_KCF,
    otherwise 1000.
method
setTrackerIdAssignmentPolicy(self, type: depthai.TrackerIdAssignmentPolicy)
Specify tracker ID assignment policy.

Parameter ``type``:
    Tracker ID assignment policy.
method
setTrackerThreshold(self, threshold: float)
Specify tracker threshold.

Parameter ``threshold``:
    Above this threshold the detected objects will be tracked. Default 0, all
    image detections are tracked.
method
setTrackerType(self, type: depthai.TrackerType)
Specify tracker type algorithm.

Parameter ``type``:
    Tracker type.
method
setTrackingPerClass(self, trackingPerClass: bool)
Whether tracker should take into consideration class label for tracking.
property
inputDetectionFrame
Input ImgFrame message on which object detection was performed. Default queue is non-blocking with size 4.
property
inputDetections
Input message with image detection from neural network. Default queue is non- blocking with size 4.
property
inputTrackerFrame
Input ImgFrame message on which tracking will be performed. RGBp, BGRp, NV12, YUV420p types are supported. Default queue is non-blocking with size 4.
property
out
Outputs Tracklets message that carries object tracking results.
property
passthroughDetectionFrame
Passthrough ImgFrame message on which object detection was performed. Suitable for when input queue is set to non-blocking behavior.
property
passthroughDetections
Passthrough image detections message from neural network output. Suitable for when input queue is set to non-blocking behavior.
property
passthroughTrackerFrame
Passthrough ImgFrame message on which tracking was performed. Suitable for when input queue is set to non-blocking behavior.

Need assistance?

Head over to Discussion Forum for technical support or any other questions you might have.