CloudSoftwareHardware
Software Stack
DepthAI
OAK Apps
AI Models

ON THIS PAGE

  • Conversion to ONNX
  • Overview
  • Conversion
  • From PyTorch
  • YOLO Models
  • From TensorFlow
  • Verification
  • Input/Output Tensors
  • Performance

Conversion to ONNX

Overview

Open Neural Network Exchange (ONNX) is a widely used format to represent machine learning models. It's open-source and supported by a wide range of frameworks and tools. Here we collect some of the most common approaches of converting your model to .onnx. It's adviced to do so as it opens up the most options at conversion for the appropriate RVC Platform.

Conversion

From PyTorch

You can utilize the PyTorch ONNX API to convert and export your model:
Python
1import torch
2# Load your PyTorch model
3your_model = Model()
4# Create a dummy input tensor matching the input shape of the model
5dummy_input = torch.randn(1, 3, 224, 224)
6# Convert and save as ONNX
7torch.onnx.export(your_model, dummy_input, 'output.onnx')
See the official tutorial for more information.

YOLO Models

If you are exporting a PyTorch (.pt) weights of a YOLO model, we recommend using our tools-cli package we prepared for just this use-case. Tools-cli is a command-line tool that simplifies the conversion process of YOLO models. It supports the conversion of YOLOs ranging from V5 through V11 and Gold Yolo including oriented bounding boxes object detection (OBB), pose estimation, and instance segmentation variants of YOLOv8 and YOLO11 to ONNX format.To install the tools-cli package, please run:
Command Line
1# Cloning the tools repository and all submodules
2git clone --recursive https://github.com/luxonis/tools.git
3# Change folder
4cd tools
5# Install the package 
6pip install .
If you are interested in running the tools through Docker, please check out this description.To convert your YOLO (e.g., yolov6n.pt) model, use this command (please note that you need to be in the root folder of the package):
Command Line
1tools yolov6n.pt --imgsz "512 288"
See the documentation for more information about the arguments.

From TensorFlow

For models in in TensorFlow (.pb), Keras (.h5), tensorflow.js (.json and .bin) or TensorFlow Lite (.tflite) format we recommend using the tf2onnx conversion tool.
  • First, install the tensorflow and tf2onnx packages:
Command Line
1pip install tensorflow
2pip install -U tf2onnx
  • Second, the conversion can be done via the command line:
Command Line
1python -m tf2onnx.convert --saved-model tensorflow-model-path --output model.onnx

Verification

Input/Output Tensors

Prior converting ONNX for RVC Platform, make sure to check that input/output tensors an ONNX model are:
  • Correctly defined (taking inputs from / sending outputs to the appropriate layers).
    • If not, use the onnx-modifier tool to re-define input/output tensors.
  • Shape is of the form NCHW (batch-size, color-channels, height, width).
    • If not, define --inputs-as-nchw data and --outputs-as-nchw flags when running the tf2onnx tool.
  • None of the shape dimensions is dynamic (e.g. that batch size is fixed).
    • If dynamic, make shapes fixed using the onnxruntime library (see this tutorial for more information).

Performance

Converting a model to ONNX can sometimes result in slight differences in model performance. These discrepancies are generally due to differences in rounding, numerical precision, layer implementations, and operation approximations. It's adviced to compare the outputs of the original and ONNX models using various test inputs. You can evaluate the differences using metrics like mean absolute error (MAE) or mean squared error (MSE).If you identify discrepancies, consider the following steps:
  • Update to the Latest ONNX Opset: Ensure you are using the latest ONNX opset version. Newer opsets often include improvements and bug fixes that enhance model compatibility.
  • Simplify Model Operations: Simplify complex operations or layers in your model before converting it to ONNX. This can help reduce the likelihood of conversion-related issues.
  • Ensure Precision Consistency: Use the same data types (e.g., float32) for weights, inputs, and outputs in both models to maintain numerical precision.
By following these steps, you can minimize discrepancies and ensure that your ONNX model closely matches the performance of your original model.