Conversion to ONNX
Overview
Open Neural Network Exchange (ONNX) is a widely used format to represent machine learning models. It's open-source and supported by a wide range of frameworks and tools. Here we collect some of the most common approaches of converting your model to.onnx
. It's adviced to do so as it opens up the most options at conversion for the appropriate RVC Platform.Conversion
From PyTorch
You can utilize the PyTorch ONNX API to convert and export your model:Python
1import torch
2# Load your PyTorch model
3your_model = Model()
4# Create a dummy input tensor matching the input shape of the model
5dummy_input = torch.randn(1, 3, 224, 224)
6# Convert and save as ONNX
7torch.onnx.export(your_model, dummy_input, 'output.onnx')
YOLO Models
If you are exporting a PyTorch (.pt) weights of a YOLO model, we recommend using our tools-cli package we prepared for just this use-case. Tools-cli is a command-line tool that simplifies the conversion process of YOLO models. It supports the conversion of YOLOs ranging from V5 through V11 and Gold Yolo including oriented bounding boxes object detection (OBB), pose estimation, and instance segmentation variants of YOLOv8 and YOLO11 to ONNX format.Please note that our tools support conversion of YOLOv9 weights only from Ultralytics.
tools-cli
package, please run:Command Line
1# Cloning the tools repository and all submodules
2git clone --recursive https://github.com/luxonis/tools.git
3# Change folder
4cd tools
5# Install the package
6pip install .
Command Line
1tools yolov6n.pt --imgsz "512 288"
From TensorFlow
For models in in TensorFlow (.pb
), Keras (.h5
), tensorflow.js (.json
and .bin
) or TensorFlow Lite (.tflite
) format we recommend using the tf2onnx conversion tool.- First, install the
tensorflow
andtf2onnx
packages:
Command Line
1pip install tensorflow
2pip install -U tf2onnx
- Second, the conversion can be done via the command line:
Command Line
1python -m tf2onnx.convert --saved-model tensorflow-model-path --output model.onnx
Verification
Input/Output Tensors
Prior converting ONNX for RVC Platform, make sure to check that input/output tensors an ONNX model are:- Correctly defined (taking inputs from / sending outputs to the appropriate layers).
- If not, use the onnx-modifier tool to re-define input/output tensors.
- Shape is of the form NCHW (batch-size, color-channels, height, width).
- If not, define
--inputs-as-nchw data
and--outputs-as-nchw
flags when running the tf2onnx tool.
- If not, define
- None of the shape dimensions is dynamic (e.g. that batch size is fixed).
- If dynamic, make shapes fixed using the onnxruntime library (see this tutorial for more information).
We suggest using the Netron tool to inspect the model.
Performance
Converting a model to ONNX can sometimes result in slight differences in model performance. These discrepancies are generally due to differences in rounding, numerical precision, layer implementations, and operation approximations. It's adviced to compare the outputs of the original and ONNX models using various test inputs. You can evaluate the differences using metrics like mean absolute error (MAE) or mean squared error (MSE).If you identify discrepancies, consider the following steps:- Update to the Latest ONNX Opset: Ensure you are using the latest ONNX opset version. Newer opsets often include improvements and bug fixes that enhance model compatibility.
- Simplify Model Operations: Simplify complex operations or layers in your model before converting it to ONNX. This can help reduce the likelihood of conversion-related issues.
- Ensure Precision Consistency: Use the same data types (e.g., float32) for weights, inputs, and outputs in both models to maintain numerical precision.