DepthAI’s Documentation¶

Note

Just received an OAK camera? Start here!

DepthAI is a Spatial AI platform, which allows robots and computers to perceive the world like a human can - what objects or features are - and where they are in physical world. It focuses on the combination of these 5 key features:

Artificial Intelligence
Computer Vision
Depth perception (Stereo, ToF)
Performant (high resolution and FPS, multiple sensors)
Embedded, low power solution

DepthAI platform is a complete ecosystem of custom hardware, firmware, and software. Best of all, it is modular and you can integrate this technology into your products (docs here).

Demo Script¶

Demo script is our multipurpose command line demo tool, built around pipeline builder API, that allows you to check DepthAI features straight from the command line - no coding required! It works USB and POE equally, automatically discovering any POE DepthAI on your LAN and/or USB DepthAI connected to your computer. If multiple are connected, it will prompt you on which to use for the demo.

To install and run the demo script on your OAK camera, type the following commands in the terminal.

git clone https://github.com/luxonis/depthai.git
cd depthai
python3 install_requirements.py
python3 depthai_demo.py

And then following up with the README.md for more usage examples. We have also prepared a step by step guide here with detailed instructions how to set up your DepthAI and run this script.

If you have issues during the installation, see our Installation page for additional OS-specific instructions

Example Use Cases¶

In this section, you’ll find an inspiration what can you build right away with DepthAI.

Pose Estimation

This example shows how to run Google Mediapipe single body pose tracking models

Created by our contributor - Geaxgx

Hand Tracking

This example shows how to run Google Mediapipe hand tracking models

Created by our contributor - Geaxgx

Human-Machine Safety

This example shows how to use the DepthAI to detect dangerous interactions between humans and other objects.

License Plates & Car Attributes Recognition

This experiment allows you to run multiple neural networks at once to collect car attributes and license plates (only Chinese)

Head posture detection

This example demonstrates 2-stage pipeline with depthai - face detection NN and head pose estimation NN

Sign Language Recognition

This example demonstrates how to recognize American Sign Language (ASL) on DepthAI using hand landmarks

Example by Cortic Technology

Gaze Estimation

This example demonstrates how to run 3 stage inference (3-series, 2-parallel) on DepthAI using Gen2 Pipeline Builder.

Original OpenVINO demo, on which this example was made, is here.

Fatigue Detection

This example demonstrates the Gen2 Pipeline Builder running face detection network and head detection network

This example was created by our partner - ArduCam

Age Gender Recognition

This example demonstrates how to run 2 stage inference on DepthAI using Gen2 Pipeline Builder.

First, a face is detected on the image and then the cropped face frame is sent to age gender recognition network, which produces the estimated results

Face Recognition

This example demonstrates the Gen2 Pipeline Builder running face detection network, head posture estimation network and face recognition network

This example was created by our partner - ArduCam

COVID-19 mask detection

This experiment allows you to run the COVID-19 mask/no-mask object detector which was trained via

the Google Colab tutorial here.

Pedestrian reidentification

This example demonstrates how to run 2 stage inference on DepthAI using Gen2 Pipeline Builder.

Original OpenVINO demo, on which this example was made, is here.

Camera Demo

This example shows how to use the DepthAI/megaAI/OAK cameras in the Gen2 Pipeline Builder over USB.

Deeplabv3 (Segmentation)

This example shows how to run Deeplabv3+ and crop the depth image based on the models output.

Fire detection

This example demonstrates the Gen2 Pipeline Builder running fire detection network

This example was created by our partner - ArduCam

Text Detection + OCR

This example demonstrates the Gen2 Pipeline Builder running text detection (EAST) followed by optical character recognition of the detected text

Tools & API Examples¶

In this section, you’ll see examples of various API usage permutations, to show what the API is capable of or to solve some meta problem, like how to stream the data, how to collect it and alike.

OCR	This pipeline implements text detection (EAST) followed by optical character recognition of the detected text
Multiple Devices	This example shows how you can use multiple DepthAI’s on a single host. The demo will find all devices connected to the host and display an RGB preview from each of them
Face recognition	Detects all faces in the frame, gets face feature vectors and compares it with database to perform face recognition
Message Syncing	This example shows how to sync messages (eg. NN results with frames) with software, based on either timestamps or sequence numbers
License plate recognition	Detects license plates and performs license plate recognition operation on the camera itself
WLS Filtering	This example demonstrates how to do host-side WLS filtering using the rectified_right and depth stream from DepthAI API
QR code scanner	QR Code detection model running on the device combined with on-host QR code decoder
Deploy Roboflow models	Deploy over 10,000 pre-trained AI models from Roboflow Universe and your own Roboflow custom models

Ecosystem¶

Core Repositories¶
depthai-python	Here you’ll find Python bindings creating the Python API of DepthAI
depthai-core	Our core API written in C++
depthai-ros	DepthAI ROS Wrapper. This is an attempt at basic DepthAI to ROS2 interface. It’s largely leveraging the existing depthai-python examples.
depthai-unity	DepthAI Unity Wrapper projects and examples. Useful for synthetic dataset generation.
depthai-hardware	This repository contains Luxonis open sourced baseboards, and contains Altium design files, documentation, and pictures to help you understand more about the embedded hardware that powers DepthAI.
depthai-ml-training	Here you can find repositories to help you connect your NN and create BLOBs.

Demo Repositories¶
depthai-experiments	In this repository, you’ll find various experiments using DepthAI. You can use those examples as a basis or a reference in your application.
depthai	This repo contains a demo application, which can load different networks, create pipelines, record video, etc. This program includes an example of depth & CNN inference and ready to use models.
depthai-core-example	CMake example project which serves as a template on how to quickly get started with C++ and depthai library
depthai-tutorials	This repo contains source code for tutorials published on docs.luxonis.com.
depthai-docker	This repository contains a Dockerfile, that allows you to run OpenVINO on DepthAI inside a Docker container.