ModelConverter

Please refer to HubAI Conversion for guidelines on the online usage of the ModelConverter tool.

Overview

ModelConverter is our open-source tool that supports conversion to all of the RVC Compiled Formats. It consists of:

ModelConverter Docker images - a set of Docker images for running conversions in a container. There are several images available for each target platform based on the required versions of the underlying conversion tools;
ModelConverter CLI - a command line interface for interracting with the ModelConverter Docker images. The CLI looks for an image with the name luxonis/modelconverter-<platform>:latest to run the conversion for the target RVC Platform.

Installation

ModelConverter requires Docker to be installed on your system. It is recommended to use Ubuntu OS for the best compatibility. On Windows or MacOS, it is recommended to install it using the Docker Desktop. Otherwise follow the installation instructions for your OS from the official website.The ModelConverter CLI can be installed from the Python Package Index (PyPI) as:

Command Line

1pip install modelconv

The ModelConverter Docker images are pulled automatically when the ModelConverter CLI commands are executed. If you do not have access to the specific image for your target RVC Platform, please check the following instructions.

Preparation

Model Source

The model of interest should be in one of the following formats:

ONNX (.onnx),
OpenVINO IR (.xml and .bin), or
TensorFlow Lite (.tflite).

The model file(s) can be standalone or packed as a NN Archive.

Calibration Data (optional)

If you do not plan to quantize your model during conversion, you can skip this step. Otherwise, prepare a directory of image files (.jpg, .png, or .jpeg) used to guide quantization. They will be automatically processed to the apporiate format by the ModelConverter. Optionally, you can also provide calibration data as .npy or .raw files. However, beware that no automatic processing is performed on these files so make sure to prepare them correctly (e.g. NHWC layout for RVC4 conversion).

Shared Folder

A shared folder must be created for file sharing between the host and the conversion containers. ModelConverter will automatically search the root for a folder named shared_with_container and mount it as /app/shared_with_container/ inside the conversion container. The folder should contain the following subfolders:

archives for storing NN Archives,
calibration_data for storing calibration data,
configs for storing configuration files,
models for storing model files,
outputs for storing conversion outputs.

Run the folder creation as:

Command Line

1mkdir shared_with_container
2cd shared_with_container
3mkdir archives calibration_data configs models outputs

Conversion

Conversion is ran using the ModelConverter CLI. The process is guided using the conversion parameters. You can define them in a Config file or provide the model source as a NN Archive and the ModelConverter will automatically fill-in the conversion parameters for you. Additionally, you can define the overrides option through CLI to change conversion parameters on the run.To run the conversion:

Prepare Config file or NN Archive for the model of interest;
Place the relevant model file(s), Config file, NN Archive file, and calibration data in the respective sub-directories of the shared_with_container folder. There is no need to provide the model file(s) in case of converting from a NN Archive. Calibration data is optional and should only be provided if you plan to quantize the model during conversion.
Run the conversion using the --path argument pointing to the Config file or NN Archive inside the mounted folder. Alternatively, s3 and GCS URLs can also provided.

Command Line

1modelconverter convert <platform> --path <url_or_path> [ conversion parameter overrides ]

Note that NN Archive contains no information about calibration data, so you must provide it manually by setting the calibration parameters in the overrides.

Examplary conversion to RVC4 using quantization:

Command Line

1modelconverter convert rvc4 --path archives/<nn_archive>.tar.xz \
2    calibration.path calibration_data/<calibration_data_dir>

Note that if you are converting using a Config file and you modified the default stages names (stages.stage_name), you have to provide the full path to each stage in the overrides. For instance, if a stage name is changed to stage1, use stages.stage1.calibration.path instead of just calibration.path

Command Line

1modelconverter convert rvc4 --path configs/<config_file>.yaml \
2    stages.stage1.calibration.path calibration_data/<calibration_data_dir>

Alternatively, you can also run the conversion without ModelConverter CLI by using the docker run command:

Command Line

1docker run --rm -it \
2    -v $(pwd)/shared_with_container:/app/shared_with_container/ \
3    luxonis/modelconverter-<platform>:latest \
4    convert <platform> \
5    --path <s3_url_or_path> [ conversion parameter overrides ]

Parameters

The following sections list all the conversion parameters with their descriptions and the respective default values.

Top-level

Parameter	Description	Default
`input_model`	Path to the model source file. Can be a local path or s3 or gcs url. Required.	-
`inputs`	List of inputs to the model.	-
`outputs`	List of outputs from the model.	-
`disable_onnx_simplification`	Disable ONNX simplification.	`False`
`keep_intermediate_outputs`	Do not delete intermediate files.	`True`

Inputs

inputs is a list of parameters for each input to the model. Each input can have the following parameters:

Parameter	Description	Default	Example
`name`	Name of the input.	-	`"input_0"`
`shape`	Shape of the input.	-	`[1, 3, 224, 224]`
`layout`	Layout of the input.	-	`"NCHW"`
`data_type`	Data type of the input.	-	`"float32"`
`encoding`	Encoding of the input. Can be one of: `RGB`, `BGR`, `GRAY`, or `NONE`	-	`"RGB"`
`encoding.from`	What encoding does the input of the source model expect.	`"RGB"`	`"RGB"`
`encoding.to`	What encoding should the exported model expect.	`"BGR"`	`"BGR"`
`mean_values`	Mean values for normalizing the input, following the channel order of the original model. Can be a single number, a list of numbers, or `"imagenet"` string for imagenet normalization.	-	`[ 123.675, 116.28, 103.53 ]`
`scale_values`	Scale values for normalizing the input, following the channel order of the original model. Can be a single number, a list of numbers, or `"imagenet"` string for imagenet normalization.	-	`[ 58.395, 57.12, 57.375 ]`
`calibration`	Specifies how to calibrate the input. For details, see Calibration Parameters	-	-

The parameters of inputs list can also be configured globally at the Top-level omitting the name parameter, in which case they will apply to all model inputs. If these parameters are also defined within the inputs list, they will override the global configuration for the specific input identified by the name parameter. If the inputs are not specified at all, they will be inferred from the model.

When encoding is set to a single value, this value is automatically applied to both encoding.from and encoding.to. To use different values, you can explicitly set encoding.from and encoding.to separately.

Outputs

Parameter	Description	Default	Example
`name`	Name of the output.	-	`"output_0"`
`shape`	Shape of the output.	-	`[1, 1000]`
`layout`	Layout of the output.	-	`"NC"`
`data_type`	Data type of the output.	-	`"float32"`

If the outputs are not specified, they will be inferred from the model.

Calibration

The calibration data can be specified in two ways:

Random

If the calibration data is not provided, the tool will generate random data for calibration. Using random calibration data is not recommended as it may lead to suboptimal quantization results. This option serves as a quick way to test the conversion process.

Parameter	Description	Default
`max_images`	Number of images to use for calibration.	`20`
`min_value`	Minimum value of the input data.	`0.0`
`max_value`	Maximum value of the input data.	`255.0`
`mean`	Mean value of the input data.	`127.5`
`std`	Standard deviation of the input data.	`35.0`
`data_type`	Data type of the input data.	`"float32"`

Guided

This calibration method uses real images to calibrate the model.The images used for calibration should be different from the images used for training the model.

Parameter	Description	Default
`path`	Path to the calibration data. Can be a local path or s3 or gcs url.	-
`max_images`	Number of images to use for calibration. The entire set is used by default.	`-1`
`resize_method`	How to resize the images, so they fit the model input shape. Can be one of: `"resize"`, `"pad"`, or `"crop"`	`"resize"`

Platform-Specific

The following parameters are specific to individual target platforms:

RVC2

Parameter	Description	Default	Example
`superblob`	Compile the model to the superblob format.	`True`	-
`compress_to_fp16`	Compress the model weights to FP16 precision.	`True`	-
`number_of_shaves`	Number of SHAVEs to use. Used only if superblob conversion is disabled.	`8`	-
`number_of_cmx_slices`	Number of CMX slices to use. Used only if superblob conversion is disabled.	`8`	-
`mo_args`	List of ddditional arguments for the Model Optimizer. Arguments provided this way will always take precedence over parameters that would otherwise be specified by the `ModelConverter`'s internal logic.	`[]`	`["--use_legacy_frontend"]`
`compile_tool_args`	List of additional arguments for the Compile Tool. Arguments provided this way will always take precedence over parameters that would otherwise be specified by the `ModelConverter`'s internal logic.	`[]`	`["-ov_api_1_0"]`

RVC3

Parameter	Description	Default	Example
`compress_to_fp16`	Compress the model weights to FP16 precision.	`True`	-
`pot_target_device`	Target device for the POT tool. Can be either `"VPU"`, or `"ANY"`.	`"VPU"`	-
`mo_args`	List of additional arguments for the Model Optimizer. Arguments provided this way will always take precedence over parameters that would otherwise be specified by the `ModelConverter`'s internal logic.	-	`["--use_legacy_frontend"]`
`compile_tool_args`	List of additional arguments for the Compile Tool. Arguments provided this way will always take precedence over parameters that would otherwise be specified by the `ModelConverter`'s internal logic.	-	`["-ov_api_1_0"]`

RVC4

Parameter	Description	Default	Example
`snpe_onnx_to_dlc_args`	List of additional arguments for the SNPE ONNX to DLC tool.	-	`["--batch", "6"]`
`snpe_dlc_quant_args`	List of additional arguments for the SNPE DLC quantization tool.	-	`["--weights_bitwidth=16"]`
`snpe_dlc_graph_prepare_args`	List of additional arguments for the SNPE DLC graph prepare tool.	-	`["--optimization_level=3"]`
`keep_raw_images`	Keep the raw images used for calibration in the intermediate data.	`False`	-
`use_per_channel_quantization`	Enables per-axis-element quantization for weights and biases in Convolution, Deconvolution, and FullyConnected ops.	`True`	-
`use_per_row_quantization`	Enables row wise quantization of Matmul and FullyConnected ops	`False`	-
`htp_socs`	List of HTP SoCs to target.	`["sm8550"]`	`["sm8550", "sm8650", "qcs6490"]`

Hailo

Parameter	Description	Default	Example
`optimization_level`	Optimization level.	`2`	-
`compression_level`	Compression level.	`2`	-
`batch_size`	Batch size.	`8`	-
`disable_compilation`	Skips the compilation of `.har` file to the final `.hef` model.	`False`	-
`alls`	List of additional custom lines added to the `alls`.	-	-
`hw_arch`	Hardware architecture.	`"hailo8"`	-

Config File

The conversion parameters can be defined in a .yaml Config file. See the illustration below or consult additional examples for more information.

Yaml

1# local path relative to the `shared_with_container` directory
2input_model: models/model.onnx
3
4mean_values: imagenet
5scale_values: imagenet
6data_type: float32
7shape: [ 1, 3, 256, 256 ]
8
9# the ONNX model expects RGB input,
10# the exported model will expect BGR input
11encoding:
12  from: RGB
13  to: BGR
14
15calibration:
16  # calibration path can be an s3 url
17  path: s3://url/to/calibration_data.zip
18  max_images: 20

ON THIS PAGE

ModelConverter

Overview

Installation

Preparation

Model Source

Calibration Data (optional)

Shared Folder

Conversion

Parameters

Top-level

Inputs

Outputs

Calibration

Random

Guided

Platform-Specific

RVC2

RVC3

RVC4

Hailo

Config File