# Conversion to ONNX

## Overview

Open Neural Network Exchange (ONNX) is a widely used format to represent machine learning models. It's open-source and supported
by a wide range of frameworks and tools. Here we collect some of the most common approaches of converting your model to .onnx.
It's adviced to do so as it opens up the most options at conversion for the appropriate RVC Platform.

## Conversion

### From PyTorch

You can utilize the [PyTorch ONNX API](https://pytorch.org/docs/stable/onnx.html) to convert and export your model:

```python
import torch
import torch.nn as nn

# Define the model architecture (must match the saved model)
class Model(nn.Module):
    ...
    
# Initialize the model and load the trained weights
model = Model()
model.load_state_dict(torch.load("model_name.pt"))
model.eval()  # Set the model to inference mode

# Define the input shape and create a dummy input tensor
input_shape = ... # e.g. (1, 3, 512, 288)
dummy_input = torch.randn(input_shape)

# Export the model to ONNX format
torch.onnx.export(model, dummy_input, "model_name.onnx")
```

This type of conversion does not alter the model’s architecture; it only changes the format in which the model is saved. As a
result, the converted model exhibits performance identical to that of the original. For more information, refer to the official
[PyTorch export tutorial](https://pytorch.org/tutorials/beginner/onnx/export_simple_model_to_onnx_tutorial.html).

#### YOLO Models

For the conversion of YOLO models, we recommend using our [tools-cli](https://github.com/luxonis/tools) package, a specialized
utility that streamlines the conversion process. The currently supported families include YOLOv5, YOLOv6, YOLOv7, YOLOv8, YOLOv9,
YOLOv10, YOLO11, YOLO12, YOLO26, YOLOE, and Gold-YOLO. For the exact set of supported YOLO families, variants, and current
limitations, refer to the upstream [supported models
table](https://github.com/luxonis/tools?tab=readme-ov-file#-supported-models).

Unlike the general PyTorch conversion, this process modifies the model architecture to standardize its outputs. This allows for
out-of-the-box parsing using the native DepthAI
[DetectionNetwork](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/detection_network.md) node. (or the
[YOLOExtendedParser](https://github.com/luxonis/depthai-nodes/tree/main/depthai_nodes/node#object-detection) parser node, for pose
and instance segmentation models).

As a result, post-processing can be performed entirely on-device, eliminating host-device data transfers and reducing overall
latency. While this is the recommended approach, you can still export YOLO models using the general conversion method. However,
keep in mind that you will need to handle output parsing manually, as native support is not provided. Additionally, be aware that
the exported ONNX model may not be directly convertible or runnable due to limitations in supported operations—both in the
converter and on the device.

You can install the tools-cli package, as:

```bash
git clone --recursive https://github.com/luxonis/tools.git
cd tools
pip install .
```

Once installed, you can run the conversion directly from the command line (just make sure you are runnig from the root of the
tools directory):

```bash
tools <MODEL>.pt --imgsz "<WIDTH> <HEIGHT>"
```

You can find more information about conversion parameters in the [tools-cli
documentation](https://github.com/luxonis/tools?tab=readme-ov-file#arguments). For example, to convert a YOLOv6n model that
expects BGR input images with a resolution of (512, 288), run the following:

```bash
tools yolov6n.pt --imgsz "512 288" --encoding BGR
```

The resulting ONNX model has input shape of (1, 3, 512, 288), and the following output shapes: (1,85,36,64), (1,85,18,32), and
(1,85,9,16) (in contrast to the original model’s output shapes as obtained by the general conversion: (1, 2304, 85), (1, 576, 85),
and (1, 144, 85); or (1, 3024, 85) if concatenated into a single tensor).

> If you are interested in running the tools using Docker, please check out these
> [guidelines](https://github.com/luxonis/tools?tab=readme-ov-file#using-docker)
> .

### From TensorFlow

For models in in TensorFlow (.pb), Keras (.h5), tensorflow.js (.json and .bin) or TensorFlow Lite (.tflite) format we recommend
using the [tf2onnx](https://github.com/onnx/tensorflow-onnx) conversion tool.

 * First, install the tensorflow and tf2onnx packages:

```bash
pip install tensorflow
pip install -U tf2onnx
```

 * Second, the conversion can be done via the command line:

```bash
python -m tf2onnx.convert --saved-model tensorflow-model-path --output model.onnx
```

## Verification

### Input/Output Tensors

Prior converting ONNX for RVC Platform, make sure to check that input/output tensors an ONNX model are:

 * Correctly defined (taking inputs from / sending outputs to the appropriate layers).
   * If not, use the [onnx-modifier](https://github.com/ZhangGe6/onnx-modifier) tool to re-define input/output tensors.
 * Shape is of the form NCHW (batch-size, color-channels, height, width).
   * If not, define --inputs-as-nchw data and --outputs-as-nchw flags when running the
     [tf2onnx](https://github.com/onnx/tensorflow-onnx) tool.
 * None of the shape dimensions is dynamic (e.g. that batch size is fixed).
   * If dynamic, make shapes fixed using the onnxruntime library (see this
     [tutorial](https://onnxruntime.ai/docs/tutorials/mobile/helpers/make-dynamic-shape-fixed.html) for more information).

> We suggest using the
> [Netron](https://netron.app/)
> tool to inspect the model.

### Performance

Converting a model to ONNX can sometimes result in slight differences in model performance. These discrepancies are generally due
to differences in rounding, numerical precision, layer implementations, and operation approximations. It's adviced to compare the
outputs of the original and ONNX models using various test inputs. You can evaluate the differences using metrics like mean
absolute error (MAE) or mean squared error (MSE).

If you identify discrepancies, consider the following steps:

 * Update to the Latest ONNX Opset: Ensure you are using the latest ONNX opset version. Newer opsets often include improvements
   and bug fixes that enhance model compatibility.
 * Simplify Model Operations: Simplify complex operations or layers in your model before converting it to ONNX. This can help
   reduce the likelihood of conversion-related issues.
 * Ensure Precision Consistency: Use the same data types (e.g., float32) for weights, inputs, and outputs in both models to
   maintain numerical precision.

By following these steps, you can minimize discrepancies and ensure that your ONNX model closely matches the performance of your
original model.