# embedding_metrics

Python API: `luxonis_train.attached_modules.metrics.embedding_metrics`

Metrics that check the embedding space of
[GhostFaceNetHead](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/nodes/heads/ghostfacenet_head.md).

[ClosestIsPositiveAccuracy](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/metrics/embedding_metrics.md)
checks whether the nearest other embedding has the same identity label.
[MedianDistances](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/metrics/embedding_metrics.md)
reports the median Euclidean distances between the embeddings. When the head sets `cross_batch_memory_size`, both metrics also
score the embeddings of earlier batches.

## Classes

### ClosestIsPositiveAccuracy

Accuracy of nearest-neighbor identity matches for embeddings.

 * `Inputs:`: * `predictions` (`Tensor`): [B, D] embeddings
    * `target` (`Tensor`): [B] identity labels
 * `Outputs:`: * `Tensor`: scalar accuracy in [0, 1]
 * `Formula:`: The metric computes the Euclidean distance between each pair of embeddings. An embedding counts only when at least
   one other embedding has the same label. The accuracy is the number of counted embeddings whose nearest other embedding has the
   same label, divided by the number of counted embeddings.

> **References**
> * Source: Converted to PyTorch from the TensorFlow code at [omoindrot.github.io/triplet-loss](https://omoindrot.github.io/triplet-loss).
 * License: Apache-2.0 (this project)

> **Notes**
> When the head sets `cross_batch_memory_size`, the metric keeps the newest embeddings of all batches, up to that number. It scores nothing until the memory is full. After that, it scores all embeddings in the memory again on each batch. The memory empties when the metric resets.

> **Example**
> Attached to a `GhostFaceNetHead` in `model.nodes`:

```yaml
- name: GhostFaceNetHead
  inputs: [GhostFaceNet]
  metrics:
    - name: ClosestIsPositiveAccuracy
```

 * `Compatible with:`: * Used by:
   [EmbeddingsModel](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/config/predefined_models/embeddings/v1/model.md)
    * Nodes:
      [GhostFaceNetHead](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/nodes/heads/ghostfacenet_head.md)

#### Methods

##### init

```python
def __init__(**kwargs):
```

Initialize the metric and register its metric states.

The method reads the type hints of the class and of its base classes. It registers each attribute whose `Annotated` hint holds a
[MetricState](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/metrics/base_metric.md)
with the `add_state` method of `torchmetrics`.
[MetricState](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/metrics/base_metric.md)
describes how the default value and the reduction of a state follow from the marker.

Parameters

 * `**kwargs`: Keyword arguments forwarded to
   [BaseAttachedModule](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/base_attached_module.md),
   such as `node`. The remaining arguments reach the `torchmetrics` `Metric`, which accepts only its own options, such as
   `sync_on_compute`, and raises `ValueError` for any other name.

Raises

 * `ValueError`: When a metric state has no `default` and its type is neither `Tensor` nor `list[...]`. Also when the `default` of
   a metric state is a list that is not empty.

##### compute

```python
def compute(self) -> Tensor:
```

Return the share of counted embeddings with a correct match.

Returns

 * `Tensor`: `correct / total`, a scalar from `0` to `1`. It is `NaN` when no embedding counted since the last reset.

##### update

```python
def update(predictions: Tensor, target: Tensor):
```

Add the nearest-neighbor matches of one batch to the counts.

Without cross-batch memory, the method scores the batch alone. With it, the method appends each embedding and its label to the
memory. It then drops the oldest entries above the memory size. While the memory holds fewer entries than the size, the method
returns and scores nothing. Otherwise it scores all embeddings in the memory.

An embedding counts only when another embedding has the same label. `correct` grows by the counted embeddings whose nearest other
embedding has the same label. `total` grows by the number of counted embeddings.

Parameters

 * `predictions` (`Tensor`): Embeddings of shape `[B, D]`, the main output of the node.
 * `target` (`Tensor`): The `metadata/id` label of each embedding, of shape `[B]`.

#### Attributes

##### correct

##### cross_batch_memory

##### node

##### supported_tasks

##### total

### MedianDistances

Median distances between embeddings, for diagnostics.

 * `Inputs:`: * `embeddings` (`Tensor`): [B, D]
    * `target` (`Tensor`): [B] identity labels
 * `Outputs:`: * `MedianDistance`, `MedianClosestDistance`, `MedianClosestPositiveDistance`,
   `MedianClosestVsClosestPositiveDistance` (`Tensor`): scalar medians
 * `Formula:`: The metric computes the Euclidean distance between each pair of embeddings. It collects four sets of distances over
   all batches and reports the median of each: * `MedianDistance`: each unordered pair of embeddings.
    * `MedianClosestDistance`: each embedding and its nearest other embedding.
    * `MedianClosestPositiveDistance`: each embedding and its nearest other embedding with the same label.
    * `MedianClosestVsClosestPositiveDistance`: the nearest same-label distance minus the nearest distance, for each embedding.
      The last two sets skip an embedding when no other embedding has its label. For an even number of values, `torch.median`
      returns the lower of the two middle values.

> **References**
> * Source: Converted to PyTorch from the TensorFlow code at [omoindrot.github.io/triplet-loss](https://omoindrot.github.io/triplet-loss).
 * License: Apache-2.0 (this project)

> **Notes**
> The cross-batch memory of the head works as in [ClosestIsPositiveAccuracy](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/metrics/embedding_metrics.md). All four values are `NaN` when the metric scored no batch. The result holds only sub-metrics, so this metric cannot be the main metric.

> **Example**
> Attached to a `GhostFaceNetHead` in `model.nodes`:

```yaml
- name: GhostFaceNetHead
  inputs: [GhostFaceNet]
  metrics:
    - name: MedianDistances
```

 * `Compatible with:`: * Used by:
   [EmbeddingsModel](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/config/predefined_models/embeddings/v1/model.md)
    * Nodes:
      [GhostFaceNetHead](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/nodes/heads/ghostfacenet_head.md)

#### Methods

##### init

```python
def __init__(**kwargs):
```

Initialize the metric and register its metric states.

The method reads the type hints of the class and of its base classes. It registers each attribute whose `Annotated` hint holds a
[MetricState](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/metrics/base_metric.md)
with the `add_state` method of `torchmetrics`.
[MetricState](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/metrics/base_metric.md)
describes how the default value and the reduction of a state follow from the marker.

Parameters

 * `**kwargs`: Keyword arguments forwarded to
   [BaseAttachedModule](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/base_attached_module.md),
   such as `node`. The remaining arguments reach the `torchmetrics` `Metric`, which accepts only its own options, such as
   `sync_on_compute`, and raises `ValueError` for any other name.

Raises

 * `ValueError`: When a metric state has no `default` and its type is neither `Tensor` nor `list[...]`. Also when the `default` of
   a metric state is a list that is not empty.

##### compute

```python
def compute(self) -> dict[str, Tensor]:
```

Return the median of each distance list.

Returns

 * `dict[str, Tensor]`: Scalar medians under the keys `"MedianDistance"`, `"MedianClosestDistance"`,
   `"MedianClosestPositiveDistance"`, and `"MedianClosestVsClosestPositiveDistance"`. All four are `NaN` when
   [update](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/metrics/embedding_metrics.md)
   added nothing since the last reset. A list that holds only empty tensors also gives `NaN`.

##### update

```python
def update(embeddings: Tensor, target: Tensor):
```

Add the distances of one batch to the four distance lists.

The cross-batch memory works as in
[ClosestIsPositiveAccuracy.update](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/attached_modules/metrics/embedding_metrics.md).
While the memory is not full, the method stores the batch and adds nothing to the lists.

The method appends one tensor to each list:

 * `all_distances`: the distance of each unordered pair.
 * `closest_distances`: the nearest distance of each embedding.
 * `positive_distances`: the nearest same-label distance of each embedding that has another embedding with its label.
 * `closest_vs_positive_distances`: the nearest same-label distance minus the nearest distance, for the same embeddings.

Parameters

 * `embeddings` (`Tensor`): Embeddings of shape `[B, D]`, the `embeddings` output of the node.
 * `target` (`Tensor`): The `metadata/id` label of each embedding, of shape `[B]`.

#### Attributes

##### all_distances

##### closest_distances

##### closest_vs_positive_distances

##### cross_batch_memory

##### node

##### positive_distances

##### supported_tasks