# ModelConverter

> Please refer to
> [HubAI Conversion](https://docs.luxonis.com/cloud/hubai/model-registry/detailed-conversion.md)
> for guidelines on the online usage of the ModelConverter tool.

## Overview

ModelConverter is our [open-source](https://github.com/luxonis/modelconverter) tool for local conversion of neural network models
to deployment formats for Luxonis devices. It consists of:

 * ModelConverter Docker images - target-specific container images that bundle the required conversion toolchains.
 * ModelConverter CLI - the modelconverter command used to run conversion, open an interactive shell, and access inference,
   benchmarking, and analysis utilities.

## Installation

ModelConverter requires Docker to be installed on your system. It is recommended to use Ubuntu OS for the best compatibility. On
Windows or macOS, it is recommended to install it using the [Docker Desktop](https://www.docker.com/products/docker-desktop).
Otherwise follow the installation instructions for your OS from the [official website](https://docs.docker.com/engine/install/).

The ModelConverter CLI can be installed from the Python Package Index (PyPI) as:

```bash
pip install modelconv
```

Run modelconverter --help to see the available commands and options.

Optional extras are available for advanced workflows:

 * pip install "modelconv[bench]" for modelconverter benchmark
 * pip install "modelconv[analysis]" for RVC4 DLC analysis and visualization

When local images are not already available, the CLI can build them automatically when possible. For target-specific image
availability and prerequisites, refer to the [repo build instructions](https://github.com/luxonis/modelconverter#local-usage).

## Preparation

### Model Source

The model of interest should be in one of the following formats:

 * ONNX (.onnx),
 * OpenVINO IR (.xml and .bin), or
 * TensorFlow Lite (.tflite).

The model file(s) can be standalone or packed as a [NN Archive](https://docs.luxonis.com/software-v3/ai-inference/nn-archive.md).

### Calibration Data (optional)

If you do not plan to quantize your model during conversion, you can skip this step. Otherwise, prepare a directory of image files
(.jpg, .png, or .jpeg) used to guide quantization. They will be automatically processed to the appropriate format by
ModelConverter. Make sure that the images used for calibration are different from the images used for training the model.
Typically, the validation dataset is used for this purpose. Optionally, you can also provide calibration data as .npy or .raw
files. However, beware that no automatic processing is performed on these files so make sure to prepare them correctly (e.g. NHWC
layout for RVC4 conversion).

### Shared Folder

When using the CLI, keep a shared_with_container directory in your current working directory. It is mounted as
/app/shared_with_container/ inside the conversion container.

The recommended directory structure is:

 * archives for storing NN Archives,
 * calibration_data for storing calibration data,
 * configs for storing configuration files,
 * models for storing model files,
 * outputs for storing conversion outputs.

Run the folder creation as:

```bash
mkdir shared_with_container
cd shared_with_container
mkdir archives calibration_data configs models outputs
```

This structure is not mandatory as long as the files are visible inside the container, but it keeps local and downloaded assets
organized. ModelConverter first checks the path exactly as provided, then resolves it relative to /app/shared_with_container/. The
output_dir can be set with --output-dir; otherwise it is autogenerated from the model name, target, and timestamp.

## Conversion

Conversion is run using the ModelConverter CLI. The process is guided using the [conversion
parameters](#ModelConverter-Parameters). You can define them in a [Config file](#ModelConverter-Parameters-Config%2520File) or
provide the model source as a [NN Archive](https://docs.luxonis.com/software-v3/ai-inference/nn-archive.md) and ModelConverter
will automatically fill in the conversion parameters for you. Additionally, you can pass key-value overrides on the CLI to change
conversion parameters for a single run.

To run the conversion:

 1. Prepare a Config file or NN Archive for the model of interest.
 2. If using local files, place the relevant model file(s), Config file, NN Archive file, and calibration data in the respective
    subdirectories of shared_with_container. There is no need to provide the model file(s) in case of converting from a NN
    Archive. Calibration data is optional and should only be provided if you plan to quantize the model during conversion.
 3. Run the conversion using the --path argument pointing to the Config file or NN Archive inside the mounted folder.
    Alternatively, s3:// and gcs:// URLs can also be provided.

```bash
modelconverter convert <platform> --path <url_or_path> [ conversion parameter overrides ]
```

### Common CLI options

| Option | Description |
| --- | --- |
| `--tool-version` | Selects the underlying conversion tool version. |
| `--image` | Uses a specific Docker image instead of the default tag lookup. |
| `--output-dir` | Sets the output directory name under `shared_with_container/outputs`. |
| `--to` | Chooses output packaging, for example `native` or `nn_archive`. |
| `--archive-preprocess` | Stores preprocessing in the NN Archive instead of baking it into the model. |

> For locally built images, the CLI expects tags such as
> `luxonis/modelconverter-rvc4:<tool-version>-latest`
> . If you use a custom tag, pass it explicitly with
> `--image`
> .

> For
> `RVC4`
> , a full SNPE build version such as
> `2.32.6.250402`
> allows the CLI to auto-download the archive on the first build if that version exists in Qualcomm's catalog. A short version
such as
> `2.32.6`
> assumes the archive or image is already available locally.

> For ONNX models larger than 2 GB, the external data file must be present next to the model as
> `<model>.onnx_data`
> . If you convert from an NN Archive, include both the
> `.onnx`
> file and the matching
> `.onnx_data`
> file inside the archive.

> Note that NN Archive contains no information about calibration data, so you must provide it manually by setting the
> `calibration`
> parameters in the
> *overrides*
> . Example conversion to RVC4 using quantization:
> ```bash
> modelconverter convert rvc4 --path archives/<nn_archive>.tar.xz \
> calibration.path calibration_data/<calibration_data_dir>
> ```

> Note that if you are converting using a Config file and you modified the
> [default stage names](https://github.com/luxonis/modelconverter/blob/main/shared_with_container/configs/defaults.yaml)
> (
> `stages.stage_name`
> ), you have to provide the full path to each stage in the
> *overrides*
> . For instance, if a stage name is changed to
> `stage1`
> , use
> `stages.stage1.calibration.path`
> instead of just
> `calibration.path`
> .
> ```bash
> modelconverter convert rvc4 --path configs/<config_file>.yaml \
> stages.stage1.calibration.path calibration_data/<calibration_data_dir>
> ```

Alternatively, you can also run the conversion without ModelConverter CLI by using the docker run command:

```bash
docker run --rm -it \
    -v $(pwd)/shared_with_container:/app/shared_with_container/ \
    luxonis/modelconverter-<platform>:<tool-version>-latest \
    convert <platform> \
    --path <s3_url_or_path> [ conversion parameter overrides ]
```

## Multi-stage conversion

ModelConverter also supports multi-stage conversion, where the output of one model becomes the input of another. This is
configured through the stages section in the Config file. For current examples, see
[defaults.yaml](https://github.com/luxonis/modelconverter/blob/main/shared_with_container/configs/defaults.yaml) and the [example
configs directory](https://github.com/luxonis/modelconverter/tree/main/shared_with_container/configs).

## Interactive shell

If you need to inspect the toolchain or run vendor tools manually inside the container, open an interactive shell:

```bash
modelconverter shell rvc4
```

## Parameters

The following sections summarize the most commonly used conversion parameters together with their default values. For the full
current set of options, see
[defaults.yaml](https://github.com/luxonis/modelconverter/blob/main/shared_with_container/configs/defaults.yaml) and the [current
example configs](https://github.com/luxonis/modelconverter/tree/main/shared_with_container/configs).

### Top-level

| Parameter | Description | Default |
| --- | --- | --- |
| `input_model` | Path to the model source file. Can be a local path or s3 or gcs url. Required. | - |
| `inputs` | List of inputs to the model. | - |
| `outputs` | List of outputs from the model. | - |
| `disable_onnx_simplification` | Disable ONNX simplification. | `False` |
| `disable_onnx_optimization` | Disable ONNX optimization. | `False` |
| `keep_intermediate_outputs` | Do not delete intermediate files. | `True` |

### Inputs

inputs is a list of parameters for each input to the model. Each input can have the following parameters:

| Parameter | Description | Default | Example |
| --- | --- | --- | --- |
| `name` | Name of the input. | - | `"input_0"` |
| `shape` | Shape of the input. | - | `[1, 3, 224, 224]` |
| `layout` | Layout of the input. | - | `"NCHW"` |
| `data_type` | Data type of the input. | - | `"float32"` |
| `encoding` | Encoding of the input. Can be one of: `RGB`, `BGR`, `GRAY`, or `NONE` | - | `"RGB"` |
| `encoding.from` | What encoding does the input of the *source* model expect. | `"RGB"` | `"RGB"` |
| `encoding.to` | What encoding should the *exported* model expect. | `"BGR"` | `"BGR"` |
| `mean_values` | Mean values for normalizing the input, following the channel order of the original model. Can be a single
number, a list of numbers, or `"imagenet"` string for imagenet normalization. | - | `[ 123.675, 116.28, 103.53 ]` |
| `scale_values` | Scale values for normalizing the input, following the channel order of the original model. Can be a single
number, a list of numbers, or `"imagenet"` string for imagenet normalization. | - | `[ 58.395, 57.12, 57.375 ]` |
| `calibration` | Specifies how to calibrate the input. For details, see [Calibration Parameters ](#Calibration%20Parameters) | -
| - |

> The parameters of
> `inputs`
> list can also be configured globally at the
> [Top-level](#ModelConverter-Parameters-Top-level)
> omitting the
> `name`
> parameter, in which case they will apply to all model inputs. If these parameters are also defined within the
> `inputs`
> list, they will override the global configuration for the specific input identified by the
> `name`
> parameter. If the inputs are not specified at all, they will be inferred from the model.

> When
> `encoding`
> is set to a single value, this value is automatically applied to both
> `encoding.from`
> and
> `encoding.to`
> . To use different values, you can explicitly set
> `encoding.from`
> and
> `encoding.to`
> separately.

### Outputs

| Parameter | Description | Default | Example |
| --- | --- | --- | --- |
| `name` | Name of the output. | - | `"output_0"` |
| `shape` | Shape of the output. | - | `[1, 1000]` |
| `layout` | Layout of the output. | - | `"NC"` |
| `data_type` | Data type of the output. | - | `"float32"` |

If the outputs are not specified, they will be inferred from the model.

### Calibration

The calibration data can be specified in two ways:

#### Random

If the calibration data is not provided, the tool will generate random data for calibration. Using random calibration data is not
recommended as it may lead to suboptimal quantization results. This option serves as a quick way to test the conversion process.

| Parameter | Description | Default |
| --- | --- | --- |
| `max_images` | Number of images to use for calibration. | `20` |
| `min_value` | Minimum value of the input data. | `0.0` |
| `max_value` | Maximum value of the input data. | `255.0` |
| `mean` | Mean value of the input data. | `127.5` |
| `std` | Standard deviation of the input data. | `35.0` |
| `data_type` | Data type of the input data. | `"float32"` |

#### Guided

This calibration method uses real images to calibrate the model.

| Parameter | Description | Default |
| --- | --- | --- |
| `path` | Path to the calibration data. Can be a local path or s3 or gcs url. | - |
| `max_images` | Number of images to use for calibration. The entire set is used by default. | `-1` |
| `resize_method` | How to resize the images, so they fit the model input shape. Can be one of: `"resize"`, `"pad"`, or `"crop"` |
`"resize"` |

### Platform-Specific

The following parameters are specific to individual target platforms:

#### RVC2

| Parameter | Description | Default | Example |
| --- | --- | --- | --- |
| `superblob` | Compile the model to the superblob format. | `True` | - |
| `compress_to_fp16` | Compress the model weights to FP16 precision. | `True` | - |
| `number_of_shaves` | Number of SHAVEs to use. Used only if superblob conversion is disabled. | `8` | - |
| `number_of_cmx_slices` | Number of CMX slices to use. Used only if superblob conversion is disabled. | `8` | - |
| `mo_args` | List of additional arguments for the Model Optimizer. Arguments provided this way will *always* take precedence over
parameters that would otherwise be specified by ModelConverter's internal logic. | `[]` | `["--use_legacy_frontend"]` |
| `compile_tool_args` | List of additional arguments for the Compile Tool. Arguments provided this way will *always* take
precedence over parameters that would otherwise be specified by the `ModelConverter`'s internal logic. | `[]` | `["-ov_api_1_0"]`
|

#### RVC4

| Parameter | Description | Default | Example |
| --- | --- | --- | --- |
| `snpe_onnx_to_dlc_args` | List of additional arguments for the SNPE ONNX to DLC tool. | - | `["--batch", "6"]` |
| `snpe_dlc_quant_args` | List of additional arguments for the SNPE DLC quantization tool. | - | `["--weights_bitwidth=16"]` |
| `snpe_dlc_graph_prepare_args` | List of additional arguments for the SNPE DLC graph prepare tool. | - | `["--htp_archs=v73"]` |
| `keep_raw_images` | Keep the raw images used for calibration in the intermediate data. | `False` | - |
| `use_per_channel_quantization` | Enables per-axis-element quantization for weights and biases in Convolution, Deconvolution, and
FullyConnected ops. | `True` | - |
| `use_per_row_quantization` | Enables row wise quantization of Matmul and FullyConnected ops | `False` | - |
| `htp_socs` | List of HTP SoCs to target. | `["sm8550"]` | `["sm8550", "sm8650", "qcs6490"]` |
| `optimization_level` | Optimization level for DLC graph preparation. Available options are `1`, `2`, and `3`. | `2` | - |
| `quantization_mode` | Pre-defined quantization modes. | `INT8_STANDARD` | - |

Quantization Modes

When overriding from the CLI, use rvc4.quantization_mode to choose between different pre-defined quantization modes for RVC4
conversions. The available modes are:

 * INT8_STANDARD: Standard INT8 quantization with calibration (default), for optimal performance (FPS) and model size.
 * INT8_ACCURACY_FOCUSED: INT8 quantization with calibration. This mode utilizes more advanced quantization techniques that may
   improve accuracy without reducing performance or increasing the model size, depending on the model.
 * INT8_INT16_MIXED: Mixed INT8 and INT16 quantization with calibration. This mode uses 8-bit weights and 16-bit activations
   across all layers for improved numeric stability and accuracy at the cost of reduced performance (FPS) and increased model
   size.
 * INT8_INT16_MIXED_ACCURACY_FOCUSED: Mixed INT8 and INT16 quantization with calibration using more advanced quantization
   techniques that may improve accuracy at the cost of reduced performance (FPS) and increased model size.
 * FP16_STANDARD: FP16 quantization without calibration, for models that require higher accuracy and numeric stability, at the
   cost of performance (FPS) and increased model size.
 * CUSTOM: Custom quantization mode, where the user can specify more advanced options in the configuration file or via
   command-line arguments.

> When
> `quantization_mode`
> is set to anything other than
> `CUSTOM`
> , the default settings for that mode will override any custom settings provided in the configuration file or via command-line
arguments (via the
> `rvc4.snpe_onnx_to_dlc_args`
> ,
> `rvc4.snpe_dlc_quant_args`
> , or
> `rvc4.snpe_dlc_graph_prepare_args`
> override options).

### Config File

The conversion parameters can be defined in a .yaml Config file. See the illustration below or consult additional
[examples](https://github.com/luxonis/modelconverter/tree/main/shared_with_container/configs) for more information.

```yaml
# local path relative to the `shared_with_container` directory
input_model: models/model.onnx

mean_values: imagenet
scale_values: imagenet
data_type: float32
shape: [ 1, 3, 256, 256 ]

# the ONNX model expects RGB input,
# the exported model will expect BGR input
encoding:
  from: RGB
  to: BGR

calibration:
  # calibration path can be an s3 url
  path: s3://url/to/calibration_data.zip
  max_images: 20
```