DepthAI’s documentation

Note

New to DepthAI? Start here!

Learn about DepthAI ecosystem, available devices, calibration instructions, and more

DepthAI is the embedded spatial AI platform built around Myriad X - a complete ecosystem of custom hardware, firmware, software, and AI training. It combines neural inference, depth vision, and feature tracking into an easy-to-use, works-in-30-seconds solution.

DepthAI offloads AI, depth vision and more - processed direct from built-in cameras - freeing your host to process application-specific data.

DepthAI gives you power of AI, depth, and tracking in a single device with a simple, easy-to-use API, written in Python and C++.

Best of all, it is modular (System on Module) and built on MIT-licensed open source hardware, affording adding these Spatial AI/CV super powers to real commercial products.

In the following sections, we will showcase what you can build with DepthAI. Different examples will showcase a different usage of DepthAI capabilities, which can both encourage you to develop your own ideas or dive deeper into DepthAI capabilities to discover them yourself.

Demo script

Demo script is our multipurpose command line demo tool, built around Gen2 Pipeline, that allows you to check DepthAI features straight from the command line - no coding required! It works USB and POE equally, automatically discovering any POE DepthAI on your LAN and/or USB DepthAI connected to your computer. If multiple are connected, it will prompt you on which to use for the demo.

Default run

To install and run the demo script on your DepthAI device, type the following commands in the terminal.

git clone git@github.com:luxonis/depthai.git
cd depthai
python3 install_requirements.py
python3 depthai_demo.py

And then following up with the README.md for more usage examples. We have also prepared a step by step guide here with detailed instructions how to set up your DepthAI and run this script.

If you have issues during the installation, see our Installation page for additional OS-specific instructions

Example Use Cases

In this section, you’ll find an inspiration what can you build right away with DepthAI

Blazepose Demo
Pose Estimation

This example shows how to run Google Mediapipe single body pose tracking models

Created by our contributor - Geaxgx

Hand tracker Demo
Hand Tracking

This example shows how to run Google Mediapipe hand tracking models

Created by our contributor - Geaxgx

Human-Machine Safety

This example shows how to use the DepthAI to detect dangerous interactions between humans and other objects.

License Plates & Car Attributes Recognition

This experiment allows you to run multiple neural networks at once to collect car attributes and license plates (only Chineese)

Head posture detection

This example demonstrates the Gen2 Pipeline Builder running face detection network and head detection network

This example was created by our partner - ArduCam

Sign Language Recognition
Sign Language Recognition

This example demonstrates how to recognize American Sign Language (ASL) on DepthAI using hand landmarks

This example was created by Cortic Technology

Gaze Estimation

This example demonstrates how to run 3 stage inference (3-series, 2-parallel) on DepthAI using Gen2 Pipeline Builder.

Original OpenVINO demo, on which this example was made, is here.

Fatigue Detection
Fatigue Detection

This example demonstrates the Gen2 Pipeline Builder running face detection network and head detection network

This example was created by our partner - ArduCam

Age Gender Recognition

This example demonstrates how to run 2 stage inference on DepthAI using Gen2 Pipeline Builder.

First, a face is detected on the image and then the cropped face frame is sent to age gender recognition network, which produces the estimated results

Face Recognition
Face Recognition

This example demonstrates the Gen2 Pipeline Builder running face detection network, head posture estimation network and face recognition network

This example was created by our partner - ArduCam

COVID-19 mask detection

This experiment allows you to run the COVID-19 mask/no-mask object detector which was trained via

the Google Colab tutorial here.

Pedestrian reidentification

This example demonstrates how to run 2 stage inference on DepthAI using Gen2 Pipeline Builder.

Original OpenVINO demo, on which this example was made, is here.

Camera Demo
Camera Demo

This example shows how to use the DepthAI/megaAI/OAK cameras in the Gen2 Pipeline Builder over USB.

Deeplabv3 (Segmentation)

This example shows how to run Deeplabv3+ on DepthAI in the Gen2 API system.

Fire detection
Fire detection

This example demonstrates the Gen2 Pipeline Builder running fire detection network

This example was created by our partner - ArduCam

Text Detection + OCR

This example demonstrates the Gen2 Pipeline Builder running text detection (EAST) followed by optical character recognition of the detected text

Tutorials

In this section, you’ll find a way to expand your knowledge in DepthAI-related topics. Tutorials listed below are a complete walkthrough in a specific topic.

  • Hello World

    Learn how to use the DepthAI Python API to display a color video stream

  • Multiple DepthAI per Host

    Learn how to use the DepthAI Python API to display a color video stream

  • Local OpenVINO Model Conversion

    Learn how to convert OpenVINO IR models into the format required to run on DepthAI

Tools & API Examples

In this section, you’ll see examples of various API usage permutations, to show what the API is capable of or to solve some meta problem, like how to stream the data, how to collect it and alike.

Tools / Examples

OCR

This pipeline implements text detection (EAST) followed by optical character recognition of the detected text

Multiple Devices

This example shows how you can use multiple DepthAI’s on a single host. The demo will find all devices connected to the host and display an RGB preview from each of them.

Class Saver

This example demonstrates how to run MobilenetSSD and collect images of detected objects, grouped by detection label

NN Frame Sync

This example shows how to present the neural network inference results on the inferenced frames

Frames Sync

This example demonstrates how to synchronize incoming frames using their sequence numbers. This allows displaying frames taken in exact same moment

Gen2 SPI

This directory contains a few examples of how to use the SPI interface with the Gen2 Pipeline builder

TensorFlow Image Classification

This example demonstrates how to run a neural network created using TensorFlow Image Classification tutorial

WLS Filtering

This example demonstrates how to do host-side WLS filtering using the rectified_right and depth stream from DepthAI Gen2 API.

Available interfaces

DepthAI API
DepthAI API (Python / C++)

Start here if you want to interact with DepthAI using programming interfaces, available both in Python and C++

DepthAI GUI
DepthAI GUI

Start here if you want to use a drag'n'drop interface to communicate with DepthAI

Ecosystem

Core Repositories

depthai-python

Here you’ll find Python bindings creating the Python API of DepthAI

depthai-core

Our core API written in C++

depthai-shared

This repository contains shared data between our main firmware and depthai-core host library.

depthai_ros2

DepthAI ROS2 Wrapper. This is an attempt at basic DepthAI to ROS2 interface. It’s largely leveraging the existing depthai python demo on luxonis/depthai.

depthai-spi-api

API of the SPI protocol

depthai-spi-library

DepthAI SPI Library

depthai-bootloader-shared

The depthai-bootloader-shared repository contains shared data between our bootloader firmware and depthai-core host library.

depthai-hardware

This repository contains Luxonis open sourced baseboards, and contains Altium design files, documentation, and pictures to help you understand more about the embedded hardware that powers DepthAI.

Tools & Docs Repositories

depthai-docs-website

If you want to contribute and update our docs, you can simply create a pull request.

depthai-gui

DepthAI GUI is a WYSIWYG tool that allows to create a custom DepthAI pipelines, run them and see the results - all in one tool.

depthai-ml-training

Here you can find repositories to help you connect your NN and create BLOBs.

depthai-tutorials

This repo contains source code for tutorials published on docs.luxonis.com.

blobconverter

Web-based tool to convert model into MyriadX blob

Factory-calibration-DepthAI

Factory Calibration (WIP); This package contains two ROS workspace one is for depthai capture and calibration node and another is for Interbotix ViperX 300 Robot Arm 6DOF (KIT-VIPX300-6DOF) arm bot control using moveit.

depthai-docker

This repository contains a Dockerfile, that allows you to run OpenVINO on DepthAI inside a Docker container.

depthai-mock

This tool allows you to record the packets produced by DepthAI device into your disk and then play them back again as they would be produced normally - but without actually running the DepthAI

sbr-util

Utility to view and manipulate SBR binary images

Demo Repositories

depthai

This repo contains a demo application, which can load different networks, create pipelines, record video, etc. This program includes an example of depth & CNN inference and ready to use models.

esp32-spi-message-demo

ESP32 reference app for interfacing with DepthAI over SPI

depthai-core-example

CMake example project which serves as a template on how to quickly get started with C++ and depthai library

Experiments Repositories

depthai-experiments

In this repository, you’ll find various experiments using DepthAI. You can use those examples as a basis or a reference in your application.

remote-monitoring

Application that allows user to report an incident when a person or a car will be detected in specified zone.

Got questions?

We’re always happy to help with code or other questions you might have.

Discord
Community Discord

Chat live with the DepthAI team and devs like you.

forum
Discussion Forum

Like chat, just asynchronous.

forum
Email Support

Send a message to our support team.