# albumentations_engine

Python API: `luxonis_ml.data.augmentations.albumentations_engine`

## Classes

### AlbumentationConfigItem

Configuration item for
[AlbumentationsEngine](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/albumentations_engine.md).

#### Attributes

##### apply_on_stages

List of pipeline stages to apply this transformation on. Valid stages are `"train"`, `"val"`, and `"test"`. By default,
transformations are applied only during the `"train"` stage.

##### name

Name of the transformation. Must be either a valid name of an [Albumentations](https://albumentations.ai/explore/) transformation,
or a name of a custom transformation registered in the
[TRANSFORMATIONS](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/custom.md)
registry.

##### params

Parameters for the transformation.

##### use_for_resizing

Whether this transformation is eligible to be used for resizing.

### AlbumentationsEngine

Augmentation engine backed by Albumentations.

Table of Contents

 * [Configuration
   Format](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/albumentations_engine.md)
 * [Transformation
   Order](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/albumentations_engine.md)
 * [Supported
   Augmentations](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/albumentations_engine.md)
   * [Albumentations
   Augmentations](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/albumentations_engine.md)
    * [Batch
      Augmentations](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/albumentations_engine.md)
      *
      [MixUp](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/albumentations_engine.md)
       * [Mosaic4](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/albumentations_engine.md)
 * [Augmenting Unsupported
   Types](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/albumentations_engine.md)
   *
   [Classification](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/albumentations_engine.md)
    * [Metadata](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/albumentations_engine.md)
    * [Arrays](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/albumentations_engine.md)
    * [Oriented Bounding
      Boxes](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/albumentations_engine.md)
 * [Custom
   Augmentations](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/albumentations_engine.md)

#### Configuration Format

The configuration contains a list of transformations, each specified by its name and optional parameters as described in the
[AlbumentationConfigItem](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/albumentations_engine.md)
schema.

The name must be either a valid name of an [Albumentations](https://albumentations.ai/explore/) transformation, or a name of a
custom transformation registered in the
[TRANSFORMATIONS](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/custom.md)
registry.

For example:

```python
[
    {
        "name": "Affine",
        "params": {
            "rotate": 30,
            "scale": 0.5,
            "p": 0.3,
        },
    },
    {
        "name": "MixUp",
        "params": {
            "alpha": [0.3, 0.7],
            "p": 0.5,
        },
    },
    {
        "name": "CustomResize",
        "use_for_resizing": True,
    },
]
```

#### Transformation Order

The order of transformations provided in the configuration is not guaranteed to be preserved. The transformations are divided into
the following groups and are applied in this order:

 1. Batch transformations: Subclasses of
    [BatchTransform](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/batch_transform.md).
 2. Spatial transformations: Subclasses of
    [A.DualTransform](https://github.com/albumentations-team/albumentations/blob/66212d77a44927a29d6a0e81621d3c27afbd929c/albumentations/core/transforms_interface.py#L545).
 3. Custom transformations: Subclasses of
    [A.BasicTransform](https://github.com/albumentations-team/albumentations/blob/66212d77a44927a29d6a0e81621d3c27afbd929c/albumentations/core/transforms_interface.py#L49),
    but not subclasses of more specific base classes above.
 4. Pixel transformations: Subclasses of
    [A.ImageOnlyTransform](https://github.com/albumentations-team/albumentations/blob/66212d77a44927a29d6a0e81621d3c27afbd929c/albumentations/core/transforms_interface.py#L756).
    These transformations act only on the image.

#### Supported Augmentations

#### Albumentations Augmentations

All augmentations provided by the Albumentations library are supported.

#### Batch Augmentations

#### MixUp

MixUp is a data augmentation technique that blends 2 source images into a single image using a weight coefficient alpha.

#### Mosaic4

Mosaic4 transformation combines 4 images into a single image by placing them in a 2×2 grid.

#### Augmenting Unsupported Types

Albumentations does not natively support all label types supported by Luxonis Data Format. This section describes how unsupported
types are handled.

Note that the following applies only to officially supported augmentations. Custom augmentations can be implemented to handle
arbitrary types.

#### Classification

Classification can be properly augmented only for multi-label tasks, where each class is tied to a bounding box. In such cases,
the classes belonging to bboxes falling outside the image are removed. In other cases, the classification annotation is kept as
is.

#### Metadata

Metadata labels can contain arbitrary data and their semantics are unknown to the augmentation engine. Therefore, the only
transformation applied to metadata is discarding metadata associated with boxes falling outside the image.

#### Arrays

Arrays are dealt with in the same way as metadata. The only transformation applied to arrays is discarding arrays associated with
bboxes falling outside the image.

#### Oriented Bounding Boxes

(Not yet implemented)

Oriented bounding boxes are of shape (N, 5) where the last element of each row contains the angle of the bbox. This format is not
supported by Albumentations, however, Albumentations support angle to be a part of the keypoints. So, the oriented bounding boxes
are split into regular bounding boxes and a set of keypoints that represent the center of the bbox and contain the angle as the
third coordinate.

Both the keypoints and the bboxes are augmented separately. At the end, the angle is extracted from the keypoints and added back
to the bounding boxes. The keypoints are discarded.

#### Custom Augmentations

Custom augmentations can be implemented by creating a subclass of
[A.BasicTransform](https://github.com/albumentations-team/albumentations/blob/66212d77a44927a29d6a0e81621d3c27afbd929c/albumentations/core/transforms_interface.py#L49)
and registering it in the
[TRANSFORMATIONS](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/custom.md)
registry.

Possible target types that the augmentation can receive are:

 * `"image"`:> The image. All augmentations should usually support this target. For subclasses of
   [A.ImageOnlyTransform](https://github.com/albumentations-team/albumentations/blob/66212d77a44927a29d6a0e81621d3c27afbd929c/albumentations/core/transforms_interface.py#L756)
   or
   [A.DualTransform](https://github.com/albumentations-team/albumentations/blob/66212d77a44927a29d6a0e81621d3c27afbd929c/albumentations/core/transforms_interface.py#L545)
   this means overriding `apply`.
 * `"bboxes"`:> Bounding boxes. For subclasses of
   [A.DualTransform](https://github.com/albumentations-team/albumentations/blob/66212d77a44927a29d6a0e81621d3c27afbd929c/albumentations/core/transforms_interface.py#L545),
   this means overriding `apply_to_bboxes`.
 * `"keypoints"`:> Keypoints. For subclasses of
   [A.DualTransform](https://github.com/albumentations-team/albumentations/blob/66212d77a44927a29d6a0e81621d3c27afbd929c/albumentations/core/transforms_interface.py#L545),
   this means overriding `apply_to_keypoints`.
 * `"mask"`:> Segmentation masks. For subclasses of
   [A.DualTransform](https://github.com/albumentations-team/albumentations/blob/66212d77a44927a29d6a0e81621d3c27afbd929c/albumentations/core/transforms_interface.py#L545),
   this means overriding `apply_to_mask`.
 * `"instance_mask"`:> Instance segmentation masks. For subclasses of
   [BatchTransform](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/batch_transform.md),
   this means overriding `apply_to_instance_mask`.
   > 
   > Subclasses of [A.DualTransform](https://github.com/albumentations-team/albumentations/blob/66212d77a44927a29d6a0e81621d3c27afbd929c/albumentations/core/transforms_interface.py#L545) do not support this target; instance masks are treated as regular masks instead.
   > 
   > Custom augmentations can support instance masks by implementing their own logic for handling them and overriding the `targets` property to include the `"instance_mask"` target.
 * `"array"`:> Arbitrary arrays. Can only be supported by custom augmentations by implementing their own logic and adding the
   `"array"` target to the `targets` property.
 * `"metadata"`:> Metadata labels. Same situation as with the `"array"` type.
 * `"classification"`:> One-hot encoded multi-task classification labels. Same situation as with the `"array"` type.

For example:

```python
class CustomArrayAugmentation(A.BasicTransform):

    @property
    @override
    def targets(self):
        return {
            "image": self.apply,
            "array": self.apply_to_array,
        }

    @override
    def apply(self, image, **kwargs):
        ...

    def apply_to_array(
        self, array: np.ndarray, **kwargs
    ) -> np.ndarray:
        ...
```

#### Methods

##### init

```python
def __init__(height: int, width: int, targets: dict[str, str], n_classes: dict[str, int], source_names: list[str], config: Iterable[Params], keep_aspect_ratio: bool = True, is_validation_pipeline: bool | None = None, pipeline_stage: Literal['train', 'val', 'test'] | None = None, min_bbox_visibility: float = 0.0, seed: int | None = None, bbox_area_threshold: float = 0.0004):
```

Create an Albumentations-backed augmentation pipeline.

Parameters

 * `height` (`int`): Target output image height.
 * `width` (`int`): Target output image width.
 * `targets` (`dict[str, str]`): Task names mapped to task types. Supported task types are `"array"`, `"classification"`,
   `"segmentation"`, `"instance_segmentation"`, `"boundingbox"`, `"keypoints"`, and metadata tasks.
 * `n_classes` (`dict[str, int]`): Number of classes per task.
 * `source_names` (`list[str]`): Source names that should be treated as image targets.
 * `config` (`Iterable[Params]`): Iterable of augmentation configuration dictionaries.
 * `keep_aspect_ratio` (`bool`): Whether the default resize transform should preserve image aspect ratio.
 * `is_validation_pipeline` (`bool | None`): Optional legacy flag selecting evaluation behavior.> **Deprecated**
   > Deprecated since version 0.5.0:use `pipeline_stage` instead.
 * `pipeline_stage` (`Literal['train', 'val', 'test'] | None`): Optional explicit pipeline stage. Valid values are `"train"`,
   `"val"`, and `"test"`.
 * `min_bbox_visibility` (`float`): Minimum visible fraction of a bounding box after augmentation.
 * `seed` (`int | None`): Optional random seed.
 * `bbox_area_threshold` (`float`): Minimum normalized bounding-box area kept after augmentation.

Raises

 * `ValueError`: If a target task type is unsupported, more than one transform is marked for resizing, or a configured transform
   is not an Albumentations transform.
 * `TypeError`: If a resizing transform has a non-numeric probability `p`.

##### apply

```python
def apply(input_batch: list[LoaderMultiOutput]) -> LoaderMultiOutput:
```

Apply the augmentation pipeline to the data.

Parameters

 * `input_batch` (`list[LoaderMultiOutput]`): Loader outputs to augment. The number of items must match the engine's batch size.

Returns

 * `LoaderMultiOutput`: Augmented loader output.

#### Attributes

##### batch_size

The batch size required by the augmentation pipeline.

The batch size is the number of images requested by the augmentation pipeline in case of batch-based augmentations.

For example, if the augmentation pipeline contains the MixUp augmentation, the batch size should be 2.

If the pipeline requires MixUp and also Mosaic4 augmentations, the batch size should be 8 = (2⋅4).

## Attributes

### Data

### TargetType