Depth and Video Sync

This example demonstrates the use of the DepthAI Sync node to synchronize output from StereoDepth and Color Camera nodes. It showcases how to process and display disparity maps from stereo cameras and video frames from a color camera in real time.

Similar samples

Demo

Setup

Please run the install script to download all required dependencies. Please note that this script must be ran from git context, so you have to download the depthai-python repository first and then run the script

Command Line

1git clone https://github.com/luxonis/depthai-python.git
2cd depthai-python/examples
3python3 install_requirements.py

For additional information, please follow the installation guide.

Source code

Python

C++

Python

GitHub

1import depthai as dai
2import numpy as np
3import cv2
4from datetime import timedelta
5
6pipeline = dai.Pipeline()
7
8monoLeft = pipeline.create(dai.node.MonoCamera)
9monoRight = pipeline.create(dai.node.MonoCamera)
10color = pipeline.create(dai.node.ColorCamera)
11stereo = pipeline.create(dai.node.StereoDepth)
12sync = pipeline.create(dai.node.Sync)
13
14xoutGrp = pipeline.create(dai.node.XLinkOut)
15
16xoutGrp.setStreamName("xout")
17
18monoLeft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_400_P)
19monoLeft.setCamera("left")
20monoRight.setResolution(dai.MonoCameraProperties.SensorResolution.THE_400_P)
21monoRight.setCamera("right")
22
23stereo.setDefaultProfilePreset(dai.node.StereoDepth.PresetMode.HIGH_ACCURACY)
24
25color.setCamera("color")
26
27sync.setSyncThreshold(timedelta(milliseconds=50))
28
29monoLeft.out.link(stereo.left)
30monoRight.out.link(stereo.right)
31
32stereo.disparity.link(sync.inputs["disparity"])
33color.video.link(sync.inputs["video"])
34
35sync.out.link(xoutGrp.input)
36
37disparityMultiplier = 255.0 / stereo.initialConfig.getMaxDisparity()
38with dai.Device(pipeline) as device:
39    queue = device.getOutputQueue("xout", 10, False)
40    while True:
41        msgGrp = queue.get()
42        for name, msg in msgGrp:
43            frame = msg.getCvFrame()
44            if name == "disparity":
45                frame = (frame * disparityMultiplier).astype(np.uint8)
46                frame = cv2.applyColorMap(frame, cv2.COLORMAP_JET)
47            cv2.imshow(name, frame)
48        if cv2.waitKey(1) == ord("q"):
49            break

How it Works

Initialize MonoCamera nodes for left and right cameras.
Set up a ColorCamera node.
Create a StereoDepth node for depth perception.
Configure the Sync node to synchronize disparity from the StereoDepth node and video frames from the ColorCamera node.
Display the synchronized frames using OpenCV. Frames are synchronized to threshold of 50 milliseconds.

Need assistance?

Head over to Discussion Forum for technical support or any other questions you might have.

ON THIS PAGE