# utils

Python API: `luxonis_train.attached_modules.metrics.mean_average_precision.utils`

Helpers of the mean average precision metrics.

[compute_metric_lists](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/metrics/mean_average_precision/utils.md)
converts a batch to the input of `torchmetrics`.
[postprocess_metrics](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/metrics/mean_average_precision/utils.md)
turns the raw results into the main value and the other values that the trainer logs.

## Functions

### add_f1_metrics

```python
def add_f1_metrics(metrics: dict[str, Tensor]) -> dict[str, Tensor]:
```

Add an F1 score for each pair of precision and recall values.

For each key that contains `"map"`, the function looks for the same key with `"mar"` in place of `"map"`. When that key exists,
the function adds the harmonic mean of the two values. The new key has `"f1"` in place of `"map"`:

F1 = (2⋅mAP⋅mAR)/(mAP + mAR)

The F1 score is `NaN` when both values are `0`. The function changes `metrics` in place.

> **Example**
> The key `"mar"` does not exist, so `"map"` gets no F1 score.

```pycon
>>> import torch
>>> metrics = {
...     "map": torch.tensor(0.4),
...     "map_small": torch.tensor(0.5),
...     "mar_small": torch.tensor(1.0),
... }
>>> result = add_f1_metrics(metrics)
>>> sorted(result)
['f1_small', 'map', 'map_small', 'mar_small']
>>> round(result["f1_small"].item(), 4)
0.6667
```

Parameters

 * `metrics` (`dict[str, Tensor]`): The metric values.

Returns

 * `dict[str, Tensor]`: The same dictionary, with the F1 scores added.

### compute_metric_lists

```python
def compute_metric_lists(boundinbox: list[Tensor], target_boundingbox: Tensor, height: int, width: int, *, masks: list[Tensor] | None = None, target_masks: Tensor | None = None) -> tuple[list[dict[str, Tensor]], list[dict[str, Tensor]]]:
```

Convert a batch of boxes to the input of `torchmetrics`.

The function builds one prediction dictionary and one target dictionary for each image of the batch. Their format is the input of
the `update` method of the `torchmetrics` `MeanAveragePrecision`:

 * A prediction dictionary holds `"boxes"`, the first four columns of the predicted boxes. It also holds `"scores"`, the fifth
   column, and `"labels"`, the sixth column as `int32`.
 * A target dictionary holds `"boxes"`, the target boxes of the image in the `xyxy` format and in pixels. It also holds
   `"labels"`, the class column as `int32`.
 * With masks, each dictionary also holds `"masks"`, the masks of the image as `bool`.

> **Example**
> ```pycon
>>> import torch
>>> boxes = [torch.tensor([[10.0, 20.0, 50.0, 60.0, 0.5, 1.0]])]
>>> targets = torch.tensor([[0.0, 1.0, 0.25, 0.5, 0.5, 0.25]])
>>> preds, target = compute_metric_lists(
...     boxes, targets, height=200, width=100
... )
>>> preds[0]["scores"].tolist(), preds[0]["labels"].tolist()
([0.5], [1])
>>> target[0]["boxes"].tolist()
[[25.0, 100.0, 75.0, 150.0]]
```

Parameters

 * `boundinbox` (`list[Tensor]`): The predicted boxes of each image, of shape `[M_i, 6]`, as `[x1, y1, x2, y2, score, class]` in pixels.
 * `target_boundingbox` (`Tensor`): The target boxes of the batch, of shape `[N, 6]`, as `[batch_index, class, x, y, w, h]`. `x` and `y` are the normalized top-left corner, and `w` and `h` are the normalized size.
 * `height` (`int`): The image height in pixels. It scales the `y` coordinates of the target boxes.
 * `width` (`int`): The image width in pixels. It scales the `x` coordinates of the target boxes.
 * `masks` (`list[Tensor] | None`): The predicted masks of each image, of shape `[M_i, H, W]`. `None` adds no masks.
 * `target_masks` (`Tensor | None`): The target masks of the batch, of shape `[N, H, W]`, one for each row of `target_boundingbox`. `None` adds no masks.

Returns

 * `tuple[list[dict[str, Tensor]], list[dict[str, Tensor]]]`: The prediction dictionaries and the target dictionaries, one of each for each item of `boundinbox`.

Raises

 * `ValueError`: When only one of `masks` and `target_masks` is given.

### postprocess_metrics

```python
def postprocess_metrics(metrics: dict[str, Tensor], class_names: Mapping[int, str], main_metric: str, device: torch.device) ->
tuple[Tensor, dict[str, Tensor]]:
```

Split the raw metric values into the main value and the others.

The function adds the F1 scores with [add_f1_metrics](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/metrics/mean_average_precision/utils.md). It then splits each per-class tensor with [process_class_metrics](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/metrics/mean_average_precision/utils.md). At the end, it removes `main_metric` from the dictionary and returns that value separately. The function changes `metrics` in place.

> **Example**
> ```pycon
>>> import torch
>>> metrics = {
...     "map": torch.tensor(0.5),
...     "map_small": torch.tensor(0.5),
...     "mar_small": torch.tensor(1.0),
...     "classes": torch.tensor([0]),
... }
>>> main, others = postprocess_metrics(
...     metrics, {0: "person"}, "map", torch.device("cpu")
... )
>>> main.item()
0.5
>>> {key: round(value.item(), 3) for key, value in others.items()}
{'map_small': 0.5, 'mar_small': 1.0, 'f1_small': 0.667}
```

Parameters

 * `metrics` (`dict[str, Tensor]`): The raw metric values. The dictionary must hold the key `"classes"`, see
   [process_class_metrics](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/metrics/mean_average_precision/utils.md).
 * `class_names` (`Mapping[int, str]`): The class names, keyed by the class index.
 * `main_metric` (`str`): The key of the main value, such as `"map"`.
 * `device` (`torch.device`): The device of the zero tensor that the function returns when `metrics` has no key `main_metric`.

Returns

 * `tuple[Tensor, dict[str, Tensor]]`: The main value and the dictionary of the other values. The main value is a zero tensor on
   `device` when `metrics` has no key `main_metric`.

### process_class_metrics

```python
def process_class_metrics(metrics: dict[str, Tensor], class_names: Mapping[int, str]) -> dict[str, Tensor]:
```

Split each per-class tensor into one scalar value per class.

The function removes the key `"classes"` and every key that ends with `"_per_class"`. A per-class tensor with more than one
element gives one new key for each class, `"<key>_<class name>"`. Its value is the element at the position of the class in
`"classes"`. A space in a class name becomes an underscore.

A per-class tensor with one element gives no new keys. For example, `torchmetrics` returns `[-1]` when the per-class metrics are
off. The function changes `metrics` in place.

> **Example**
> ```pycon
>>> import torch
>>> metrics = {
...     "map": torch.tensor(0.5),
...     "map_per_class": torch.tensor([0.25, 0.75]),
...     "classes": torch.tensor([0, 1]),
... }
>>> names = {0: "person", 1: "traffic light"}
>>> result = process_class_metrics(metrics, names)
>>> sorted(result)
['map', 'map_per_class_person', 'map_per_class_traffic_light']
>>> result["map_per_class_traffic_light"].item()
0.75
```

Parameters

 * `metrics` (`dict[str, Tensor]`): The metric values. The dictionary must hold the key `"classes"`, a tensor with the class indices in the order of the per-class values.
 * `class_names` (`Mapping[int, str]`): The class names, keyed by the class index. It must hold each index of `"classes"`.

Returns

 * `dict[str, Tensor]`: The same dictionary, without `"classes"` and the per-class tensors, and with the value of each class.