ModelConverter

Please refer to HubAI Conversion for guidelines on the online usage of the ModelConverter tool.

Overview

ModelConverter is our open-source tool for local conversion of neural network models to deployment formats for Luxonis devices. It consists of:

ModelConverter Docker images - target-specific container images that bundle the required conversion toolchains.
ModelConverter CLI - the modelconverter command used to run conversion, open an interactive shell, and access inference, benchmarking, and analysis utilities.

Installation

ModelConverter requires Docker to be installed on your system. It is recommended to use Ubuntu OS for the best compatibility. On Windows or macOS, it is recommended to install it using the Docker Desktop. Otherwise follow the installation instructions for your OS from the official website.The ModelConverter CLI can be installed from the Python Package Index (PyPI) as:

Command Line

1pip install modelconv

Run modelconverter --help to see the available commands and options.Optional extras are available for advanced workflows:

pip install "modelconv[bench]" for modelconverter benchmark
pip install "modelconv[analysis]" for RVC4 DLC analysis and visualization

When local images are not already available, the CLI can build them automatically when possible. For target-specific image availability and prerequisites, refer to the repo build instructions.

Preparation

Model Source

The model of interest should be in one of the following formats:

ONNX (.onnx),
OpenVINO IR (.xml and .bin), or
TensorFlow Lite (.tflite).

The model file(s) can be standalone or packed as a NN Archive.

Calibration Data (optional)

If you do not plan to quantize your model during conversion, you can skip this step. Otherwise, prepare a directory of image files (.jpg, .png, or .jpeg) used to guide quantization. They will be automatically processed to the appropriate format by ModelConverter. Make sure that the images used for calibration are different from the images used for training the model. Typically, the validation dataset is used for this purpose. Optionally, you can also provide calibration data as .npy or .raw files. However, beware that no automatic processing is performed on these files so make sure to prepare them correctly (e.g. NHWC layout for RVC4 conversion).

Shared Folder

When using the CLI, keep a shared_with_container directory in your current working directory. It is mounted as /app/shared_with_container/ inside the conversion container.The recommended directory structure is:

archives for storing NN Archives,
calibration_data for storing calibration data,
configs for storing configuration files,
models for storing model files,
outputs for storing conversion outputs.

Run the folder creation as:

Command Line

1mkdir shared_with_container
2cd shared_with_container
3mkdir archives calibration_data configs models outputs

This structure is not mandatory as long as the files are visible inside the container, but it keeps local and downloaded assets organized. ModelConverter first checks the path exactly as provided, then resolves it relative to /app/shared_with_container/. The output_dir can be set with --output-dir; otherwise it is autogenerated from the model name, target, and timestamp.

Conversion

Conversion is run using the ModelConverter CLI. The process is guided using the conversion parameters. You can define them in a Config file or provide the model source as a NN Archive and ModelConverter will automatically fill in the conversion parameters for you. Additionally, you can pass key-value overrides on the CLI to change conversion parameters for a single run.To run the conversion:

Prepare a Config file or NN Archive for the model of interest.
If using local files, place the relevant model file(s), Config file, NN Archive file, and calibration data in the respective subdirectories of shared_with_container. There is no need to provide the model file(s) in case of converting from a NN Archive. Calibration data is optional and should only be provided if you plan to quantize the model during conversion.
Run the conversion using the --path argument pointing to the Config file or NN Archive inside the mounted folder. Alternatively, s3:// and gcs:// URLs can also be provided.

Command Line

1modelconverter convert <platform> --path <url_or_path> [ conversion parameter overrides ]

Common CLI options

Option	Description
`--tool-version`	Selects the underlying conversion tool version.
`--image`	Uses a specific Docker image instead of the default tag lookup.
`--output-dir`	Sets the output directory name under `shared_with_container/outputs`.
`--to`	Chooses output packaging, for example `native` or `nn_archive`.
`--archive-preprocess`	Stores preprocessing in the NN Archive instead of baking it into the model.

For locally built images, the CLI expects tags such as luxonis/modelconverter-rvc4:<tool-version>-latest. If you use a custom tag, pass it explicitly with --image.

For RVC4, a full SNPE build version such as 2.32.6.250402 allows the CLI to auto-download the archive on the first build if that version exists in Qualcomm's catalog. A short version such as 2.32.6 assumes the archive or image is already available locally.

For ONNX models larger than 2 GB, the external data file must be present next to the model as <model>.onnx_data. If you convert from an NN Archive, include both the .onnx file and the matching .onnx_data file inside the archive.

Note that NN Archive contains no information about calibration data, so you must provide it manually by setting the calibration parameters in the overrides.

Example conversion to RVC4 using quantization:

Command Line

1modelconverter convert rvc4 --path archives/<nn_archive>.tar.xz \
2    calibration.path calibration_data/<calibration_data_dir>

Note that if you are converting using a Config file and you modified the default stage names (stages.stage_name), you have to provide the full path to each stage in the overrides. For instance, if a stage name is changed to stage1, use stages.stage1.calibration.path instead of just calibration.path.

Command Line

1modelconverter convert rvc4 --path configs/<config_file>.yaml \
2    stages.stage1.calibration.path calibration_data/<calibration_data_dir>

Alternatively, you can also run the conversion without ModelConverter CLI by using the docker run command:

Command Line

1docker run --rm -it \
2    -v $(pwd)/shared_with_container:/app/shared_with_container/ \
3    luxonis/modelconverter-<platform>:<tool-version>-latest \
4    convert <platform> \
5    --path <s3_url_or_path> [ conversion parameter overrides ]

Multi-stage conversion

ModelConverter also supports multi-stage conversion, where the output of one model becomes the input of another. This is configured through the stages section in the Config file. For current examples, see defaults.yaml and the example configs directory.

Interactive shell

If you need to inspect the toolchain or run vendor tools manually inside the container, open an interactive shell:

Command Line

1modelconverter shell rvc4

Parameters

The following sections summarize the most commonly used conversion parameters together with their default values. For the full current set of options, see defaults.yaml and the current example configs.

Top-level

Parameter	Description	Default
`input_model`	Path to the model source file. Can be a local path or s3 or gcs url. Required.	-
`inputs`	List of inputs to the model.	-
`outputs`	List of outputs from the model.	-
`disable_onnx_simplification`	Disable ONNX simplification.	`False`
`disable_onnx_optimization`	Disable ONNX optimization.	`False`
`keep_intermediate_outputs`	Do not delete intermediate files.	`True`

Inputs

inputs is a list of parameters for each input to the model. Each input can have the following parameters:

Parameter	Description	Default	Example
`name`	Name of the input.	-	`"input_0"`
`shape`	Shape of the input.	-	`[1, 3, 224, 224]`
`layout`	Layout of the input.	-	`"NCHW"`
`data_type`	Data type of the input.	-	`"float32"`
`encoding`	Encoding of the input. Can be one of: `RGB`, `BGR`, `GRAY`, or `NONE`	-	`"RGB"`
`encoding.from`	What encoding does the input of the source model expect.	`"RGB"`	`"RGB"`
`encoding.to`	What encoding should the exported model expect.	`"BGR"`	`"BGR"`
`mean_values`	Mean values for normalizing the input, following the channel order of the original model. Can be a single number, a list of numbers, or `"imagenet"` string for imagenet normalization.	-	`[ 123.675, 116.28, 103.53 ]`
`scale_values`	Scale values for normalizing the input, following the channel order of the original model. Can be a single number, a list of numbers, or `"imagenet"` string for imagenet normalization.	-	`[ 58.395, 57.12, 57.375 ]`
`calibration`	Specifies how to calibrate the input. For details, see Calibration Parameters	-	-

The parameters of inputs list can also be configured globally at the Top-level omitting the name parameter, in which case they will apply to all model inputs. If these parameters are also defined within the inputs list, they will override the global configuration for the specific input identified by the name parameter. If the inputs are not specified at all, they will be inferred from the model.

When encoding is set to a single value, this value is automatically applied to both encoding.from and encoding.to. To use different values, you can explicitly set encoding.from and encoding.to separately.

Outputs

Parameter	Description	Default	Example
`name`	Name of the output.	-	`"output_0"`
`shape`	Shape of the output.	-	`[1, 1000]`
`layout`	Layout of the output.	-	`"NC"`
`data_type`	Data type of the output.	-	`"float32"`

If the outputs are not specified, they will be inferred from the model.

Calibration

The calibration data can be specified in two ways:

Random

If the calibration data is not provided, the tool will generate random data for calibration. Using random calibration data is not recommended as it may lead to suboptimal quantization results. This option serves as a quick way to test the conversion process.

Parameter	Description	Default
`max_images`	Number of images to use for calibration.	`20`
`min_value`	Minimum value of the input data.	`0.0`
`max_value`	Maximum value of the input data.	`255.0`
`mean`	Mean value of the input data.	`127.5`
`std`	Standard deviation of the input data.	`35.0`
`data_type`	Data type of the input data.	`"float32"`

Guided

This calibration method uses real images to calibrate the model.

Parameter	Description	Default
`path`	Path to the calibration data. Can be a local path or s3 or gcs url.	-
`max_images`	Number of images to use for calibration. The entire set is used by default.	`-1`
`resize_method`	How to resize the images, so they fit the model input shape. Can be one of: `"resize"`, `"pad"`, or `"crop"`	`"resize"`

Platform-Specific

The following parameters are specific to individual target platforms:

RVC2

Parameter	Description	Default	Example
`superblob`	Compile the model to the superblob format.	`True`	-
`compress_to_fp16`	Compress the model weights to FP16 precision.	`True`	-
`number_of_shaves`	Number of SHAVEs to use. Used only if superblob conversion is disabled.	`8`	-
`number_of_cmx_slices`	Number of CMX slices to use. Used only if superblob conversion is disabled.	`8`	-
`mo_args`	List of additional arguments for the Model Optimizer. Arguments provided this way will always take precedence over parameters that would otherwise be specified by ModelConverter's internal logic.	`[]`	`["--use_legacy_frontend"]`
`compile_tool_args`	List of additional arguments for the Compile Tool. Arguments provided this way will always take precedence over parameters that would otherwise be specified by the `ModelConverter`'s internal logic.	`[]`	`["-ov_api_1_0"]`

RVC4

Parameter	Description	Default	Example
`snpe_onnx_to_dlc_args`	List of additional arguments for the SNPE ONNX to DLC tool.	-	`["--batch", "6"]`
`snpe_dlc_quant_args`	List of additional arguments for the SNPE DLC quantization tool.	-	`["--weights_bitwidth=16"]`
`snpe_dlc_graph_prepare_args`	List of additional arguments for the SNPE DLC graph prepare tool.	-	`["--htp_archs=v73"]`
`keep_raw_images`	Keep the raw images used for calibration in the intermediate data.	`False`	-
`use_per_channel_quantization`	Enables per-axis-element quantization for weights and biases in Convolution, Deconvolution, and FullyConnected ops.	`True`	-
`use_per_row_quantization`	Enables row wise quantization of Matmul and FullyConnected ops	`False`	-
`htp_socs`	List of HTP SoCs to target.	`["sm8550"]`	`["sm8550", "sm8650", "qcs6490"]`
`optimization_level`	Optimization level for DLC graph preparation. Available options are `1`, `2`, and `3`.	`2`	-
`quantization_mode`	Pre-defined quantization modes.	`INT8_STANDARD`	-

Quantization ModesWhen overriding from the CLI, use rvc4.quantization_mode to choose between different pre-defined quantization modes for RVC4 conversions. The available modes are:

INT8_STANDARD: Standard INT8 quantization with calibration (default), for optimal performance (FPS) and model size.
INT8_ACCURACY_FOCUSED: INT8 quantization with calibration. This mode utilizes more advanced quantization techniques that may improve accuracy without reducing performance or increasing the model size, depending on the model.
INT8_INT16_MIXED: Mixed INT8 and INT16 quantization with calibration. This mode uses 8-bit weights and 16-bit activations across all layers for improved numeric stability and accuracy at the cost of reduced performance (FPS) and increased model size.
INT8_INT16_MIXED_ACCURACY_FOCUSED: Mixed INT8 and INT16 quantization with calibration using more advanced quantization techniques that may improve accuracy at the cost of reduced performance (FPS) and increased model size.
FP16_STANDARD: FP16 quantization without calibration, for models that require higher accuracy and numeric stability, at the cost of performance (FPS) and increased model size.
CUSTOM: Custom quantization mode, where the user can specify more advanced options in the configuration file or via command-line arguments.

When quantization_mode is set to anything other than CUSTOM, the default settings for that mode will override any custom settings provided in the configuration file or via command-line arguments (via the rvc4.snpe_onnx_to_dlc_args, rvc4.snpe_dlc_quant_args, or rvc4.snpe_dlc_graph_prepare_args override options).

Config File

The conversion parameters can be defined in a .yaml Config file. See the illustration below or consult additional examples for more information.

Yaml

1# local path relative to the `shared_with_container` directory
2input_model: models/model.onnx
3
4mean_values: imagenet
5scale_values: imagenet
6data_type: float32
7shape: [ 1, 3, 256, 256 ]
8
9# the ONNX model expects RGB input,
10# the exported model will expect BGR input
11encoding:
12  from: RGB
13  to: BGR
14
15calibration:
16  # calibration path can be an s3 url
17  path: s3://url/to/calibration_data.zip
18  max_images: 20

ON THIS PAGE

ModelConverterView as Markdown

Overview

Installation

Preparation

Model Source

Calibration Data (optional)

Shared Folder

Conversion

Common CLI options

Multi-stage conversion

Interactive shell

Parameters

Top-level

Inputs

Outputs

Calibration

Random

Guided

Platform-Specific

RVC2

RVC4

Config File

ModelConverter