# model

Python API: `luxonis_train.config.predefined_models.instance_segmentation.v1.model`

The instance segmentation model.

## Classes

### InstanceSegmentationModel

Instance segmentation, which predicts a mask for each detected object.

 * `Throughput:`: Frames per second at 384x512. * `light`: 39 on RVC2, 223 on RVC4
    * `medium`: 19 on RVC2, 184 on RVC4
    * `heavy`: 7 on RVC2, 107 on RVC4

> **Example**
> The `model` section of a config:

```yaml
model:
  predefined_model:
    name: InstanceSegmentationModel
    params:
      variant: light
```

 * `Components:`: * Nodes:
   [EfficientRep](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/nodes/backbones/efficientrep/efficientrep.md)
   ->
   [RepPANNeck](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/nodes/necks/reppan_neck/reppan_neck.md)
   ->
   [PrecisionSegmentBBoxHead](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/nodes/heads/precision_seg_bbox_head.md)
    * Losses:
      [PrecisionDFLSegmentationLoss](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/losses/precision_dfl_segmentation_loss.md)
    * Metrics: *
      [ConfusionMatrix](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/metrics/confusion_matrix/confusion_matrix.md)
       * [MeanAveragePrecision](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/metrics/mean_average_precision/mean_average_precision.md)
    * Visualizers:
      [InstanceSegmentationVisualizer](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/visualizers/instance_segmentation_visualizer.md)
    * Main metric:
      [MeanAveragePrecision](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/metrics/mean_average_precision/mean_average_precision.md)
    * Variants: * `light`
       * `medium`
       * `heavy`

#### Methods

##### init

```python
def __init__(**kwargs):
```

Initialize the model from the names of its components.

All arguments are keyword-only. The `typechecked` decorator checks their types at run time.

> **Notes**
> A `freezing` key in `backbone_params`, `neck_params`, or `head_params` holds the [FreezingConfig](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/config/config.md) of that node, either as an instance or as a dictionary of its fields. A dictionary without `active` freezes the node. [nodes](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/config/predefined_models/base_predefined_model.md) removes the key when it builds the configs.

Parameters

 * `backbone`: The class name of the registered backbone node.
 * `backbone_variant`: The variant of the backbone. `None` builds the backbone without variant parameters.
 * `head`: The class name of the registered head node.
 * `head_variant`: The variant of the head. `None` builds the head without variant parameters.
 * `neck`: The class name of the registered neck node. `None` connects the head to the backbone.
 * `neck_variant`: The variant of the neck. `None` builds the neck without variant parameters.
 * `loss`: The class name of the registered loss. The model attaches it to the head with weight `1.0`.
 * `metrics`: The class names of the registered metrics attached to the head. A string names one metric. `None` attaches no
   metric.
 * `main_metric`: The metric to mark as the main metric. The trainer keeps the checkpoints with the highest values of this metric
   in `best_val_metric`. `None` takes the only name in `metrics`, or no metric when `metrics` is empty. A name that is not in
   `metrics` marks no metric. When no metric of the config is marked,
   [ModelConfig.check_main_metric](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/config/config.md)
   marks the first one.
 * `visualizer`: The class name of the registered visualizer attached to the head. `None` attaches no visualizer.
 * `confusion_matrix_available`: Whether the head supports the
   [ConfusionMatrix](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/metrics/confusion_matrix/confusion_matrix.md)
   metric. A subclass sets it for its head.
 * `backbone_params`: The constructor parameters of the backbone. A `freezing` key does not reach the constructor. See the notes.
 * `neck_params`: The constructor parameters of the neck, with the same `freezing` key.
 * `use_neck`: Build the neck. `False` leaves the neck out even when `neck` is set, and the head reads from the backbone.
 * `head_params`: The constructor parameters of the head, with the same `freezing` key.
 * `loss_params`: The constructor parameters of the loss.
 * `metrics_params`: The constructor parameters that every metric in `metrics` receives. The `ConfusionMatrix` metric that
   `enable_confusion_matrix` adds does not receive them.
 * `visualizer_params`: The constructor parameters of the visualizer.
 * `enable_confusion_matrix`: Attach the `ConfusionMatrix` metric to the head, without the main metric flag. It has no effect when
   `confusion_matrix_available` is `False`.
 * `confusion_matrix_params`: The constructor parameters of the `ConfusionMatrix` metric.
 * `task_name`: The dataset task the head reads. It becomes the `task_name` of the head node.
 * `torchmetrics_task`: A value for the `torchmetrics_task` key that every metric in `metrics` receives. `None` adds no key. The
   key goes into `metrics_params`, so a non-empty `metrics_params` dictionary changes in place. No metric of this package reads
   the key. The
   [TorchMetricWrapper](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/metrics/torchmetrics.md)
   metrics read `task` and pass `torchmetrics_task` on to `torchmetrics`, which raises `ValueError` for it. Set `task` in
   `metrics_params` for them instead.
 * `per_class_metrics`: A value for the `per_class_metrics` key that every metric in `metrics` receives. `None` adds no key. When
   [LuxonisLightningModule](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/lightning/luxonis_lightning.md)
   builds a metric, the key becomes the per-class parameter that the metric class declares. When the class declares none, the
   module drops the key and logs a warning.
 * `finetuning`: The finetuning entries of each component. Each dictionary becomes a
   [FinetuningConfig](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/config/config.md)
   of that node.
 * `**kwargs`

Raises

 * `ValueError`: When `main_metric` is `None` and `metrics` names more than one metric.

##### get_variants

```python
def get_variants() -> tuple[str, dict[str, Params]]:
```

Get the default variant name and the available variants.

The default is `light`. Each variant sets `backbone_variant` and `neck_variant` to one size:

 * `light`: `"n"`
 * `medium`: `"s"`
 * `heavy`: `"l"`

Each variant also sets `weights` to `"download"` in `backbone_params` and `neck_params`. Both nodes then download and load the
COCO checkpoint of their variant. The variants do not set `head_params`, so the head starts without a checkpoint. A
`backbone_params` or `neck_params` given in the config replaces the whole dictionary of the variant. Set `weights` in it again to
keep the COCO checkpoint.

> **Example**
> ```pycon
>>> default, variants = InstanceSegmentationModel.get_variants()
>>> default
'light'
>>> variants["heavy"]["neck_variant"]
'l'
>>> variants["heavy"]["neck_params"]
{'weights': 'download'}
```

Returns

 * `tuple[str, dict[str, Params]]`: `"light"` and the three variants with their constructor arguments.