OAK & Raspberry Pi

Our devices can also be connected to small SBCs like a Raspberry Pi (RPi). To install DepthAI on a RPi, you should install dependencies and then install the library with pip. DepthAI library has prebuilt wheels for RPi on piwheels.

Using pre-configured RPi OS image

We provide Raspberry Pi OS images that have DepthAI and all its dependencies pre-installed. Images are available here.

  • OAK_CM4_POE - Uses RPi OS (default one), has pre-installed DepthAI. Required hardware: 8 GB uSD card or larger. This image is flashed on uSD cards that get sent with OAK-D-CM3, OAK-D-CM4, OAK-D CM4 PoE. To enable 2nd uSD card, see workaround here. We recommend that users use this image for all OAK cameras.

  • KS_Rpi_OS_Full - Uses RPi Full OS, has pre-installed DepthAI and CEP. Required hardware: 32 GB uSD card or larger, RPi 4 with 4 GB RAM. This image was flashed on uSD cards that were sent to KickStarter backers. Only use this image if you want CEP platform!

Steps taken to create these images and changelog can be found here.

Flashing uSD card

  1. Download the RPi image from Google Drive (link above). Download Balena Etcher program. You can also use RPi Imager, but we have noticed that sometimes there are issues.

  2. Open Balena Etcher. If you have downloaded .AppImage, you might need to right-click on it and click Run to start the program.

  3. Click on the Flash from file and select the .zip image you have previously downloaded.

  4. Select the target uSD card.

  5. Click Flash! and wait until it’s finished.

  6. After flashing is complete, safely remove the uSD card, insert it into RPi, and power the RPi

Powering OAK from RPi

On RPi documentation, it states that Raspberry Pi 2 or later models can supply up to 1.2A through a USB port. Even at maximum workload, OAK cameras draw less than 1A, which means that you can power an OAK camera directly from RPi via USB-C cable, except if:

  • You have additional USB devices connected to a RPi USB port that draw too much current. In total, RPi can supply 1.2A, so if all USB devices draw more than 1.2A, brownout will occur.

  • You have the Pro version of the OAK camera that has IR laser dot projector and IR illumination LED, which additionally draw up to 0.25A.

You can externally power your OAK camera either via barrel jack (on older devices), or with a Y-adapter.

SSH into the RPi

If you are using a pre-configured RPi OS image, SSH server is already enabled, otherwise you need to create file ssh inside this boot volume, so SSH server will be enabled on startup (tutorial here).

../../_images/boot_volume.png

If you have RPi with WiFi connectivity, you can set the WiFi SSID/Password by creating (and editing) wpa_supplicant.conf inside the boot volume (tutorial here).

After connecting RPi via ethernet/WiFi to the same LAN as your computer, you can SSH into the RPi with the command below. If RPi and computer are not in the same LAN, you will have to specify RPi’s IP instead of luxonis.local - which is the default name of your RPi on pre-configured RPi images.

-X argument enables X11 forwarding on Linux - so OpenCV’s imshow images will get forwarded to your computer via SSH. On MacOS, X11 isn’t installed by default, so you have to download it, more information here.

When you SSH into the RPi for the first time, it will warn you about authenticity, like the log below.

The authenticity of host 'raspberrypi.local (192.168.1.222)' can't be established.
ECDSA key fingerprint is SHA256:stb5mbRQeX6veOq8Wzg81rz9IHonxJR2++Q8bDYryTo.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

type yes to continue.

After this command, you will need to provide the password. The default RPi OS password is raspberry. You should change it.

After SSH was successful, let’s try running an example app python depthai-python/examples/MobileNet/rgb_mobilenet.py

../../_images/works.png

Known image limitations

  1. On OAK_CM4_PoE, since image V8, second uSD card support is disabled. That’s because if uSD card support is enabled (for storage purposes), WiFi connectivity is disabled. To enable 2nd uSD card support (and disable WiFi connectivity), you can edit /boot/config.txt and uncomment the line dtoverlay=sdhost,poll_once=off at the end of the file, then reboot the system.

Flashing an image to the eMMC

OAK-D-CM4 and OAK-D CM4 PoE have Raspberry Pi Compute module 4 (CM4) on-board. Most OAK-D-CM4 batches have CM4 with an eMMC memory on there, and most OAK-D CM4 PoE boards have CM4 Lite, which doesn’t have eMMC memory, but instead boots from the uSD card. It’s straight forward how to flash an image to the uSD card, and here we will go through the process of flashing an image to the eMMC that’s on the CM4.

Here’s an OAK-D-CM4, which has CM4 with eMMC memory that is indicated by the red arrow. CM4 Lite versions won’t have this chip in that location.

../../_images/cm4-emmc.png

To flash an image to the CM4 eMMC, we first need to enable USB_BOOT with the header pin. By default, connector is on the 2 header pins that are indicated as DIS (disabled), so we have to move the connector to the EN pins (enabled).

../../_images/usb-boot.png

After switching the connector, we can connect a micro-USB cable from our computer to the micro-USB connector that’s also indicated by the red box above. The OAK device (CM4, to be specific) also has to be powered via 5V barrel jack.

After connecting both the micro-USB and power, we need to enable RPi USB boot. We have followed the tutorial here. After building the program and running it, you should see something similar:

/Documents/usbboot$ sudo ./rpiboot
RPIBOOT: build-date Apr 21 2022 version 20220315~121405 445356e1
Waiting for BCM2835/6/7/2711...
Loading embedded: bootcode4.bin
Sending bootcode.bin
Successful read 4 bytes
Waiting for BCM2835/6/7/2711...
Loading embedded: bootcode4.bin
Second stage boot server
Loading embedded: start4.elf
File read: start4.elf
Second stage boot server done

After the boot is done, you should see the eMMC memory inside Raspberry Pi Imager or Balena Etcher (what we have used, screenshot below), so you can easily flash desired image to the eMMC. After flashing is complete, make sure to disable the USB boot (by switching the connector again) and restart the device. It should boot from the newly flashed image!

../../_images/etcher.png

Raspberry Pi Zero

Raspberry Pi Zero is a great single-board computer (SBC) in terms of price per performance. You can get one for about $5. It has 512MB RAM, 1GHz single-core CPU, and optional on-board Bluetooth and WiFi connectivity (RPI Zero W, which costs about $10). It consumes about 0.5W.

../../_images/rpi_zero.png

RPi Zero is sufficient for streaming metadata (NN results), low-resolution encoded videos, or light computing. If you intend to stream high-res videos, perform heavy computing (e.g. with OpenCV) or post-processing, RPi Zero won’t be the best choice for host computer.

Connect OAK camera to a RPI Zero

RPI Zero has one micro USB connector for peripherals. You can use your standard USB-C (to USB-A) cable, but you will need OTG connector/cable to connect an OAK camera to the RPI Zero.

../../_images/rpi_zero_otg.png

All of the above options work. We are not connected/affiliated with these companies, but here’s where we got the left connector (Aliexpress), and right connector (Aliexpress).

../../_images/zero_oak_d.png

From our testing, RPi Zero was able to provide enough power to the OAK-D without an external power supply (either power jack or Y-adapter).

RPi Troubleshooting

The Raspberry Pi has a max limit of 1.2A across all its USB ports, and OAK cameras can take up to 1A (at max power, usually closer to 500mA).

So if you are seeing lockups, it could be that you are over this 1.2A limit as a result of the total power of the USB devices drawing from the Pi. Using a powered hub can prevent this, or powering fewer other things off of the Pi over USB.

This may also manifest in OAK randomly crashing on the Pi. This can become particularly often if OAK is configured to do many things at once. This is becoming increasingly likely as we enable DepthAI to do more and more things at once - and thereby increase the maximum power that OAK can pull. It is seeming like the peak power (current) spikes from OAK can go over what the Pi can handle, causing OAK to brown-out and return an error.

So if you are experiencing issues with DepthAI stability on Raspberry, try powering OAK camera via a power-supply and/or a powered USB hub to see if the error goes away.

If you SSH into your RPi without -x argument and try to display a frame (eg. cv2.imshow()), you will likely get an error like below:

  • (-2:Unspecified error) Can't initialize GTK backend in function 'cvInitSystem'

  • qt.qpa.xcb: could not connect to display. qt.qpa.plugin: Could not load the Qt platform plugin "xcb".

  • This application failed to start because no Qt platform plugin could be initialized.

The solution is to use X11 forwarding when SSHing to the RPi, see SSH into the RPi.

Got questions?

We’re always happy to help with code or other questions you might have.