Getting started with OAK IoT devices¶
Robotics Vision Core 2 is the SoC+circuitry on the OAK camera, where you can run your pipeline
Robotics Vision Core 2 is connected to the host (eg. PC)
On ESP32 you can run post-processing of the messages and optionally send the results to a server (if connected to a WiFi network) or to a Bluetooth device (eg. a smartphone)
Usually, developers would develop their solution with the device connected via USB-C, and use potentially use OTA updates via ESP32 in production.
Currently, the bottleneck for some use cases is the SPI throughput, which is about 1MB/s (8Mbps). This means that you can stream frames over the SPI, just not high-resolution / unencoded streams.
Common use cases for the ESP32¶
Sending metadata results (spatial coordinates, NN results, tracklets etc.)
Capturing and streaming video/images
System information logging
Sending data to the cloud (eg. AWS/Azure/GCP or any other IOT platform)
Sending data to a Bluetooth device (eg. smartphone)
Explained in more detail:
You could run spatial object detections and/or object tracking pipeline on the VPU and send tracklets with spatial data over the SPI to the ESP32. On ESP32 you could run some simple filtering and/or NN result decoding and then send final results to the cloud
ESP32 can also flash DepthAI bootloader and/or pipeline, which means OTA (over-the-air) updates are supported.
Pipeline on the VPU could be configured to send system information (using SystemLogger node) to the ESP32 which would forward them to a logging platform
ESP32 also has Bluetooth capabilities, so you could forward the data from the ESP32 to a smartphone
How to get started with the development¶
Install the ESP-IDF, instructions here.
After building, you can flash your ESP32 using
idf.py -p PORT flash monitor(replace
PORTwith the ESP32 port, eg.
/dev/ttyUSB0). You might need to change the permission of the port with
sudo chmod 777 PORTso idf.py can access it.
After flashing the ESP32, you can start the pipeline. If you have used a demo ESP32 code, you should run the corresponding python script that’s inside the demo folder.
We’re always happy to help with code or other questions you might have.