# HubAI

## Overview

HubAI utilizes
[ModelConverter](https://docs.luxonis.com/software-v3/ai-inference/conversion/rvc-conversion/offline/modelconverter.md) in the
cloud. It's a convenient way to convert your model to deployment formats for Luxonis devices. Please consult the [HubAI
Conversion](https://docs.luxonis.com/cloud/hubai/model-registry/detailed-conversion.md) guidelines for more information.

To perform the conversion directly in Python using the [HubAI SDK](https://github.com/luxonis/hubai-sdk) package, refer to the
instructions below.

For installation and more information about the HubAI SDK, please refer to the [HubAI
SDK](https://docs.luxonis.com/cloud/hubai/model-registry/hubai-sdk.md) documentation.

## Quick Start

In order to use the online conversion, you first need to obtain an API key from [HubAI](https://hub.luxonis.com/ai). You can do
this by signing up for a free account and generating an API key in the [team
settings](https://docs.luxonis.com/cloud/api/api-keys.md).

Once you have the API key, you can run the conversion using the following code:

```python
import os

from hubai_sdk import HubAIClient

api_key = os.getenv("HUBAI_API_KEY")
client = HubAIClient(api_key=api_key)

converted_model = client.convert.RVC2(
    path="path/to/model.onnx",
    name="my-converted-model",
)
```

## Python API

The Python API for online conversion is available under the HubAIClient.convert namespace. Specific conversion functions for
individual targets are accessible under the target name, for example convert.RVC2 and convert.RVC4.

There is also a generic conversion entrypoint if you want to choose the target programmatically:

```python
from hubai_sdk.utils.types import Target

response = client.convert.convert(
    target=Target.RVC2,
    path="path/to/model.onnx",
    name="my-converted-model",
)
```

The conversion function takes a number of parameters to specify the model and the conversion options. Here we list the main ones
to get you started, but consider checking out the [full API
documentation](https://github.com/luxonis/hubai-sdk/blob/main/docs/available_parameters.md).

### General Parameters

General parameters applicable to all conversion functions.

| argument | type | description |
| --- | --- | --- |
| `path` | `str` | The path to the model file. |
| `tool_version` | `str \| None` | The version of the conversion tool. |
| `quantization_mode` | `Literal["INT8_STANDARD", "INT8_ACCURACY_FOCUSED", "INT8_INT16_MIXED", "FP16_STANDARD", "FP32_STANDARD"]`
| Quantization or precision preset used by targets that support it, for example `INT8_STANDARD` or `FP16_STANDARD`. |
| `output_dir` | `str \| None` | The directory to save the converted model. If not specified, the model will be saved in the
current working directory. |

### YOLO Parameters

These parameters are only relevant if you're converting a YOLO model.

| argument | type | description |
| --- | --- | --- |
| `yolo_input_shape` | `list[int] \| None` | The input shape of the YOLO model. |
| `yolo_version` | `str \| None` | YOLO version (for example `"yolov8"`). |
| `yolo_class_names` | `list[str] \| None` | The class names of the model. |

### RVC2 Parameters

Parameters specific to the RVC2 conversion.

| argument | type | description |
| --- | --- | --- |
| `mo_args` | `list[str] \| None` | The arguments to pass to the model optimizer. |
| `compile_tool_args` | `list[str] \| None` | The arguments to pass to the BLOB compiler. |
| `compress_to_fp16` | `bool` | Whether to compress the model's weights to FP16 precision. Defaults to `True`. |
| `number_of_shaves` | `int` | The number of shaves to use. Defaults to `8`. |
| `superblob` | `bool` | Whether to create a superblob. Defaults to `True`. Use `False` if you want legacy RVC2 blob conversion. |

### RVC4 Parameters

Parameters specific to the RVC4 conversion.

| argument | type | description |
| --- | --- | --- |
| `snpe_onnx_to_dlc_args` | `list[str] \| None` | The arguments to pass to the `snpe-onnx-to-dlc` tool. |
| `snpe_dlc_quant_args` | `list[str] \| None` | The arguments to pass to the `snpe-dlc-quant` tool. |
| `snpe_dlc_graph_prepare_args` | `list[str] \| None` | The arguments to pass to the `snpe-dlc-graph-prepare` tool. |
| `use_per_channel_quantization` | `bool` | Whether to use per-channel quantization. Defaults to `True`. |
| `use_per_row_quantization` | `bool` | Whether to use per-row quantization. Defaults to `False`. |
| `htp_socs` | `list[str] \| None` | The list of HTP SoCs to use. |

### HubAI Resource Parameters

The conversion API can also create or attach HubAI resources while converting. Common parameters include:

| argument | type | description |
| --- | --- | --- |
| `model_id` | `str \| None` | Use an existing model instead of creating a new one. |
| `name` | `str \| None` | Name of the model to create. If omitted, the SDK uses the model file stem. |
| `license_type` | `str \| None` | License metadata for the HubAI model resource. |
| `tasks` | `list[str] \| None` | Task metadata for the HubAI model resource. |
| `variant_id` | `str \| None` | Use an existing model variant instead of creating a new one. |
| `variant_version` | `str \| None` | Version identifier for the created variant. |
| `input_shape` | `list[int] \| None` | Input shape metadata for the created model instance. |

For the detailed overview of model, variant, and model-instance parameters, refer to the [full API
documentation](https://github.com/luxonis/hubai-sdk/blob/main/docs/available_parameters.md).

## CLI Reference

The conversion can also be done using the command line interface. Start with:

```bash
hubai login
hubai convert RVC2 --path /path/to/model.onnx --name "my-model"
```

See hubai convert --help for the full list of options.

## Migration from BlobConverter

[BlobConverter](https://pypi.org/project/blobconverter/) is our previous library for converting models to the BLOB format for
earlier OAK conversion workflows. This library is being replaced by modelconverter and HubAI SDK, which eventually become the only
supported way of converting models in the future.

blobconverter is still available and can be used for conversion, but we recommend using HubAI SDK for new projects. The API of
HubAI SDK is similar to that of blobconverter, but there are some differences in the parameters and the way the conversion is
done.

blobconverter offers several functions for converting models from different frameworks, such as from_onnx, from_openvino, and
from_tf. These functions are now replaced by the convert.RVC2 function in HubAI SDK, which takes a single argument path that
specifies the path to the model file.

The following table shows the mapping between the parameters of blobconverter and HubAI SDK. The parameters are grouped by their
purpose. The first column shows the parameters of blobconverter, the second column shows the equivalent parameters in HubAI SDK,
and the third column contains additional notes.

| `blobconverter` | `HubAI SDK` | Notes |
| --- | --- | --- |
| `model` | `path` | The model file path. |
| `xml` | `path` | The XML file path. Only for conversion from OpenVINO IR. |
| `bin` | `opts["input_bin"]` | The BIN file path. Only for conversion from OpenVINO IR. |
| `version` | `tool_version` | The version of the conversion tool. |
| `data_type` | `quantization_mode` | The quantization mode of the model. |
| `shaves` | `number_of_shaves` | The number of shaves to use. |
| `optimizer_params` | `mo_args` | The arguments to pass to the model optimizer. |
| `compile_params` | `compile_tool_args` | The arguments to pass to the BLOB compiler. |

By default, HubAI SDK has superblob enabled which is only supported on DepthAI v3. If you want to convert a model to legacy RVC2
format (blob), you can pass superblob=False to the convert.RVC2 function.

### Simple Conversion

Simple ONNX conversion using blobconverter

```python
import blobconverter

blob = blobconverter.from_onnx(
    model="resnet18.onnx",
)
```

Equivalent code using HubAI SDK

```python
response = client.convert.RVC2(
    path="resnet18.onnx",
)

blob = response.downloaded_path
```

### Conversion from OpenVINO IR

blobconverter example

```python
import blobconverter

blob = blobconverter.from_openvino(
    xml="resnet18.xml",
    bin="resnet18.bin",
)
```

HubAI SDK example

```python
# When the XML and BIN files are at the same location,
# only the XML needs to be specified
response = client.convert.RVC2("resnet18.xml")
blob = response.downloaded_path

# Otherwise, the BIN file can be specified using
# the `opts` parameter
response = client.convert.RVC2(
    path="resnet18.xml",
    opts={
        "input_bin": "resnet18.bin",
    }
)
blob = response.downloaded_path
```

### Conversion from TFLite

> `HubAI`
> online conversion does not support conversion from frozen PB files, only TFLITE files are supported.

blobconverter

```python
import blobconverter

blob = blobconverter.from_tf(
    frozen_pb="resnet18.tflite",
)
```

Equivalent code using HubAI SDK

```python
response = client.convert.RVC2(
    path="resnet18.tflite",

)

blob = response.downloaded_path
```

### Advanced Parameters

blobconverter.from_onnx with advanced parameters

```python
import blobconverter

blob = blobconverter.from_onnx(
    model="resnet18.onnx",
    data_type="FP16",
    version="2021.4",
    shaves=6,
    optimizer_params=[
        "--mean_values=[127.5,127.5,127.5]",
        "--scale_values=[255,255,255]",
    ],
    compile_params=["-ip U8"],

)
```

Equivalent code using HubAI SDK

```python
response = client.convert.RVC2(
    path="resnet18.onnx",
    quantization_mode="FP16_STANDARD",
    tool_version="2021.4.0",
    number_of_shaves=6,
    mo_args=[
        "mean_values=[127.5,127.5,127.5]",
        "scale_values=[255,255,255]"
    ],
    compile_tool_args=["-ip", "U8"],
)

blob = response.downloaded_path
```

### Caffe Conversion

Conversion from the Caffe framework is not supported.

## Migration from tools.luxonis.com

Tools web app hosted on [tools.luxonis.com](https://tools.luxonis.com/) is our legacy application for easily converting YOLO based
object detection models. This is now being replaced by HubAI online conversion, which will eventually become the only supported
way of converting models in the future.

Compared to the web application, HubAI online conversion also adds support for converting instance segmentation, pose, oriented
detection, and classification YOLO models. For the current conversion capabilities, refer to the [HubAI
Conversion](https://docs.luxonis.com/cloud/hubai/model-registry/detailed-conversion.md) documentation.

Here is an example of how to easily convert an existing .pth file to a RVC2 compatible .superblob format.

```python
from hubai_sdk import HubAIClient

client = HubAIClient(api_key=api_key)

converted_model = client.convert.RVC2(
    path="yolov6n.pt",
    yolo_version="yolov6r4",
)
```

This will use our Hub cloud service to make a conversion, and you'll get a .tar.xz file with all the relevant files downloaded
when the conversion finishes. Consider checking the [YOLO
parameters](https://docs.luxonis.com/software-v3/ai-inference/conversion/rvc-conversion/online/hubai.md) section for other
available flags.