Getting started with OAK IoT devices

Note

This tutorial is intended for OAK-D-IoT-75 and OAK-D-IoT-40, which both have an ESP32 on the board.

Code examples

Code examples can be found in our esp32-spi-message-demo github repository.

Overview

../../_images/iot-diagram.png

Overview explained:

Usually, developers would develop their solution with the device connected via USB-C, and use potentially use OTA updates via ESP32 in production.

Limitations

Currently, the bottleneck for some use cases is the SPI throughput, which is about 1MB/s (8Mbps). This means that you can stream frames over the SPI, just not high-resolution / unencoded streams.

As of now we only support ESP-IDF as the development framework. In the future, we might support Arduino.

Common use cases for the ESP32

TL;DR:

  • Sending metadata results (spatial coordinates, NN results, tracklets etc.)

  • OTA updates

  • Capturing and streaming video/images

  • System information logging

  • Sending data to the cloud (eg. AWS/Azure/GCP or any other IOT platform)

  • Sending data to a Bluetooth device (eg. smartphone)

Explained in more detail:

  • You could run spatial object detections and/or object tracking pipeline on the VPU and send tracklets with spatial data over the SPI to the ESP32. On ESP32 you could run some simple filtering and/or NN result decoding and then send final results to the cloud

  • ESP32 can also flash DepthAI bootloader and/or pipeline, which means OTA (over-the-air) updates are supported.

  • We have an example for both sending a whole image and part of an image through the SPI from the VPU to the ESP32

  • Pipeline on the VPU could be configured to send system information (using SystemLogger node) to the ESP32 which would forward them to a logging platform

  • ESP32 also has Bluetooth capabilities, so you could forward the data from the ESP32 to a smartphone

How to get started with the development

  1. Install the ESP-IDF, instructions here.

  2. After setting up the environmental variables, you can build any demo from the esp32-spi-message-demo repository with idf.py build.

  3. After building, you can flash your ESP32 using idf.py -p PORT flash monitor (replace PORT with the ESP32 port, eg. /dev/ttyUSB0). You might need to change the permission of the port with sudo chmod 777 PORT so idf.py can access it.

  4. After flashing the ESP32, you can start the pipeline. If you have used a demo ESP32 code, you should run the corresponding python script that’s inside the demo folder.

Got questions?

We’re always happy to help with code or other questions you might have.