Local OpenVINO Model Conversion¶
In this tutorial, you’ll learn how to convert OpenVINO IR models into the format required to run on DepthAI, even on a low-powered Raspberry Pi. I’ll introduce you to the OpenVINO toolset, the Open Model Zoo (where we’ll download the face-detection-retail-0004 model), and show you how to generate the files needed to run model inference on your DepthAI board.
Haven’t heard of OpenVINO or the Open Model Zoo? I’ll start with a quick introduction of why we need these tools.
What is OpenVINO?¶
Under-the-hood, DepthAI uses the Intel technology to perform high-speed model inference. However, you can’t just dump your neural net into the chip and get high-performance for free. That’s where OpenVINO comes in. OpenVINO is a free toolkit that converts a deep learning model into a format that runs on Intel Hardware. Once the model is converted, it’s common to see Frames Per Second (FPS) improve by 25x or more. Are a couple of small steps worth a 25x FPS increase? Often, the answer is yes!
What is the Open Model Zoo?¶
The Open Model Zoo is a library of freely-available pre-trained models. The Zoo also contains scripts for downloading those models into a compile-ready format to run on DepthAI.
DepthAI is able to run many of the object detection models in the Zoo.
If you have OpenVINO installed or want to follow official installation, skip this step.
Please note that the following install instructions are for Ubuntu 18.04 OS, if you intend to use other OS, follow the official OpenVINO installation
DepthAI requires OpenVINO version
2020.1. Let’s get a package for our OS and meeting this version with the following command:
apt-get update apt-get install -y software-properties-common add-apt-repository -y ppa:deadsnakes/ppa apt-get update apt-get install -y wget pciutils python3.8 libpng-dev libcairo2-dev libpango1.0-dev libglib2.0-dev libgtk2.0-dev libswscale-dev libavcodec-dev libavformat-dev cd mkdir openvino_install && cd openvino_install wget http://registrationcenter-download.intel.com/akdlm/irc_nas/16345/l_openvino_toolkit_p_2020.1.023.tgz tar --strip-components=1 -zxvf l_openvino_toolkit_p_2020.1.023.tgz ./install_openvino_dependencies.sh ./install.sh # when finished, you can go ahead and do "rm -r ~/openvino_install"
Now, first screen we’ll wee is EULA, just hit
Enter, scroll through and type
Next one is agreement to Intel Software Improvement Program, it’s not relevant so you can choose whether consent (
or not (
Next, you may see the Missing Prerequisites screen showing that
Intel® Graphics Compute Runtime for OpenCL™ Driver is missing - you can go ahead and ignore this warning.
Finally, we’ll see the install summary - please verify that it has a correct location pointed out -
If all looks good, go ahead and proceed (
1). If the missing prerequisites screen appears again, feel free to skip it.
Let’s verify that a correct version is installed on your host. Check your version by running the following from a terminal session:
You should see output similar to:
Thu Jan 23 19:14:14 MSK 2020 d349c3ba4a2508be72f413fa4dee92cc0e4bc0e1 releases_2020_1_InferenceEngine_37988
Verify that you see
releases_2020_1 in your output. If you do, move on. If you are on a different version,
goto the OpenVINO site and download the
2020.1 version for your OS:
Check if the Model Downloader is installed¶
When installing OpenVINO, you can choose to perform a smaller install to save disk space. This custom install may not include the model downloader script. Lets check if the downloader was installed. In a terminal session, type the following:
find /opt/intel/ -iname downloader.py
Move on if you see the output below:
Didn’t see any output? Don’t fret if
downloader.py isn’t found. We’ll install this below.
Install Open Model Zoo Downloader¶
If the downloader tools weren’t found, we’ll install the tools by cloning the Open Model Zoo Repo and installing the tool dependencies.
Start a terminal session and run the following commands in your terminal:
apt-get install -y git curl curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py python3 get-pip.py rm get-pip.py cd ~ git clone https://github.com/opencv/open_model_zoo.git cd open_model_zoo git checkout tags/2020.1 cd tools/downloader python3 -m pip install --user -r ./requirements.in
This clones the repo into a
~/open_model_zoo directory, checks out the required
2020.1 version, and
installs the downloader dependencies.
Create an OPEN_MODEL_DOWNLOADER environment variable¶
Typing the full path to
downloader.py can use a lot of keystrokes. In an effort to extend your keyboard life,
let’s store the path to this script in an environment variable.
Run the following in your terminal:
export OPEN_MODEL_DOWNLOADER='INSERT PATH TO YOUR downloader.py SCRIPT'
INSERT PATH TO YOUR downloader.py SCRIPT can be found via:
find /opt/intel/ -iname downloader.py find ~ -iname downloader.py
For example, if you installed
Download the face-detection-retail-0004 model¶
We’ve installed everything we need to download models from the Open Model Zoo! We’ll now use the
Model Downloader to download the
face-detection-retail-0004 model files. Run the following in your terminal:
$OPEN_MODEL_DOWNLOADER --name face-detection-retail-0004 --output_dir ~/open_model_zoo_downloads/
This will download the model files to
~/open_model_zoo_downloads/. Specifically, the model files we need are
You’ll see two files within the directory:
$ ls -lh total 1.3M -rw-r--r-- 1 root root 1.2M Jul 28 12:40 face-detection-retail-0004.bin -rw-r--r-- 1 root root 100K Jul 28 12:40 face-detection-retail-0004.xml
The model is in the OpenVINO Intermediate Representation (IR) format:
face-detection-retail-0004.xml- Describes the network topology
face-detection-retail-0004.bin- Contains the weights and biases binary data.
This means we are ready to compile the model for the MyriadX!
Compile the model¶
The MyriadX chip used on our DepthAI board does not use the IR format files directly. Instead, we need to generate
Let’s find where
myriad_compile is located. In your terminal, run:
find /opt/intel/ -iname myriad_compile
You should see the output similar to this
find /opt/intel/ -iname myriad_compile /opt/intel/openvino_2020.1.023/deployment_tools/inference_engine/lib/intel64/myriad_compile
Since it’s such a long path, let’s store the
myriad_compile executable in an environment variable (just like
export MYRIAD_COMPILE=$(find /opt/intel/ -iname myriad_compile)
Activate OpenVINO environment¶
In order to use
myriad_compile tool, we need to activate our OpenVINO environment.
First, let’s find
find /opt/intel/ -name "setupvars.sh" /opt/intel/openvino_2020.1.023/opencv/setupvars.sh /opt/intel/openvino_2020.1.023/bin/setupvars.sh
We’re interested in
bin/setupvars.sh file, so let’s go ahead and source it to activate the environment:
source /opt/intel/openvino_2020.1.023/bin/setupvars.sh [setupvars.sh] OpenVINO environment initialized
If you see
[setupvars.sh] OpenVINO environment initialized then your environment should be initialized correctly
$MYRIAD_COMPILE -m ~/open_model_zoo_downloads/intel/face-detection-retail-0004/FP16/face-detection-retail-0004.xml -ip U8 -VPU_MYRIAD_PLATFORM VPU_MYRIAD_2480 -VPU_NUMBER_OF_SHAVES 4 -VPU_NUMBER_OF_CMX_SLICES 4
You should see:
Inference Engine: API version ............ 2.1 Build .................. 37988 Description ....... API Done
Where’s the blob file? It’s located in the same folder as
ls -lh ~/open_model_zoo_downloads/intel/face-detection-retail-0004/FP16/ total 2.6M -rw-r--r-- 1 root root 1.2M Jul 28 12:40 face-detection-retail-0004.bin -rw-r--r-- 1 root root 1.3M Jul 28 12:50 face-detection-retail-0004.blob -rw-r--r-- 1 root root 100K Jul 28 12:40 face-detection-retail-0004.xml
Run and display the model output¶
With neural network
blob in place, we’re ready to roll!
To verify that the model is running correctly, let’s modify a bit the program we’ve created in Hello World tutorial
In particular, let’s change the
setBlobPath invocation to load our model. Remember to replace the paths to correct ones that you have!
- detection_nn.setBlobPath("/path/to/mobilenet-ssd.blob") - detection_nn.setBlobPath("/path/to/face-detection-retail-0004.blob")
And that’s all!
You should see output annotated output similar to:
Reviewing the flow¶
The flow we walked through works for other pre-trained object detection models in the Open Model Zoo:
Download the model:
$OPEN_MODEL_DOWNLOADER --name [INSERT MODEL NAME] --output_dir ~/open_model_zoo_downloads/
Create the MyriadX blob file:
$MYRIAD_COMPILE -m [INSERT PATH TO MODEL XML FILE] -ip U8 -VPU_MYRIAD_PLATFORM VPU_MYRIAD_2480 -VPU_NUMBER_OF_SHAVES 4 -VPU_NUMBER_OF_CMX_SLICES 4
Use this model in your script
You’re on your way! You can find the complete code for this tutorial on GitHub.
We’re always happy to help with code or other questions you might have.