# luxonis_loader_torch

Python API: `luxonis_train.loaders.luxonis_loader_torch`

The default loader, which reads a `LuxonisDataset`.

[LuxonisLoaderTorch](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/loaders/luxonis_loader_torch.md)
opens a dataset that exists, or parses a directory into a new dataset.

## Classes

### LuxonisLoaderTorch

The default loader, which reads a `LuxonisDataset`.

The loader wraps a `LuxonisLoader` from `luxonis_ml`. The `LuxonisLoader` reads the images and the labels of the splits in the
view, and applies the augmentations. This class converts the arrays to tensors. It can also change the class order and the
keypoint order of each task.

> **Example**
> The `loader` section of a config that reads an existing dataset:

```yaml
loader:
  name: LuxonisLoaderTorch
  params:
    dataset_name: coco_test
```

The `loader` section of a config that parses a directory into a new dataset:

```yaml
loader:
  name: LuxonisLoaderTorch
  params:
    dataset_dir: data/my_dataset
    dataset_name: my_dataset
```

#### Methods

##### init

```python
def __init__(dataset_name: str | None = None, dataset_dir: str | None = None, dataset_type: DatasetType | None = None, team_id: str | None = None, bucket_type: Literal['internal', 'external'] = 'internal', bucket_storage: Literal['local', 's3', 'gcs', 'azure'] = 'local', update_mode: Literal['all', 'missing'] = 'all', delete_existing: bool = True, filter_task_names: list[str] | None = None, min_bbox_visibility: float = 0.0, bbox_area_threshold: float = 0.0004, class_order_per_task: dict[str, list[str]] | None = None, kpts_mapping_per_task: dict[str, list[int]] | None = None, return_sample_metadata: bool = False, **kwargs):
```

Initialize the dataset and the `LuxonisLoader`.

With `dataset_dir`, the loader parses the directory into a new dataset, or opens an existing local dataset, see `delete_existing`.
Without it, the loader opens the dataset `dataset_name`. The `LuxonisLoader` downloads a remote dataset when it starts.

Parameters

 * `dataset_name` (`str | None`): The name of the dataset. Without `dataset_dir`, the loader opens this dataset. With
   `dataset_dir`, the parsed dataset gets this name; `None` uses the directory name.
 * `dataset_dir` (`str | None`): The directory to parse, in a format that `LuxonisParser` recognizes. It can be a local path, a
   remote URL, or a ZIP file. The parser downloads a remote directory to `data/` in the working directory.
 * `dataset_type` (`DatasetType | None`): The format of `dataset_dir`. `None` lets the parser detect it.
 * `team_id` (`str | None`): The team ID of the dataset. It selects its local and remote location. `None` uses the
   `LUXONISML_TEAM_ID` setting of `luxonis_ml`.
 * `bucket_type` (`Literal['internal', 'external']`): The bucket type of a remote dataset. The loader uses it only without
   `dataset_dir`.
 * `bucket_storage` (`Literal['local', 's3', 'gcs', 'azure']`): The storage backend of the dataset.
 * `update_mode` (`Literal['all', 'missing']`): The sync mode for the media files of a remote dataset. `"all"` downloads all media
   files again. `"missing"` downloads only the media files that are not local. For a remote dataset, the `LuxonisLoader` always
   downloads the annotations and the metadata.
 * `delete_existing` (`bool`): What to do when `dataset_dir` is set and a local dataset with the same name exists. `True` logs a
   warning, deletes the existing dataset, and parses the directory again. `False` opens the existing dataset and does not parse.
 * `filter_task_names` (`list[str] | None`): The names of the tasks to load. `None` loads all tasks. A name that is not in the
   dataset makes `LuxonisLoader` raise `ValueError`.
 * `min_bbox_visibility` (`float`): The minimum fraction of a box that must stay visible after the augmentations.
 * `bbox_area_threshold` (`float`): The minimum area of a box, relative to the image area, in `[0, 1]`. The augmentations remove a
   smaller box and the labels of its instance, such as the keypoints and the instance mask.
 * `class_order_per_task` (`dict[str, list[str]] | None`): The class names of each task in their desired order. Each list must
   contain exactly the classes of its task. `None` keeps the dataset order.
 * `kpts_mapping_per_task` (`dict[str, list[int]] | None`): A new keypoint order for each task. For a list `m`, the keypoint at
   position `j` is the original keypoint `m[j]`. Each list must map every keypoint of its task. `None` keeps the original order.
 * `return_sample_metadata` (`bool`): Whether `__getitem__` returns the sample metadata as a third element.
 * `**kwargs`: Arguments for
   [BaseLoaderTorch](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/loaders/base_loader.md),
   such as `view`, `height`, and `width`. This loader needs `view`, and values other than `None` for `height`, `width`, and
   `augmentation_config`.

Raises

 * `ValueError`: If both `dataset_dir` and `dataset_name` are `None`, or if `height`, `width`, or `augmentation_config` is `None`.
 * `KeyError`: If `kpts_mapping_per_task` has a task that is not in the dataset, or a task without keypoint labels.

##### augment_test_image

```python
def augment_test_image(img: dict[str, Tensor] | Tensor) -> Tensor:
```

Apply the augmentations of the loader to one raw image.

Inference calls this method to prepare an image like the samples of the view. The augmentations get one sample with no labels.
They include the resize to `height` and `width`.

Parameters

 * `img` (`dict[str, Tensor] | Tensor`): The image of shape `[H, W, C]`. A dictionary maps each source name to its image. A tensor
   is the image of `image_source`.

Returns

 * `Tensor`: The augmented image of shape `[height, width, C]`. For a dictionary with more than one source, it is the first image
   of the augmented dictionary. When the `LuxonisLoader` has no augmentations, the method returns the `image_source` image with no
   change. The `LuxonisLoader` that `__init__` builds always has augmentations.

##### get_categorical_encodings

```python
def get_categorical_encodings(self) -> dict[str, dict[str, int]]:
```

Return the integer code of each category of each metadata label.

The codes come from the dataset metadata. The labels of the loader hold these codes in place of the category names.

Returns

 * `dict[str, dict[str, int]]`: The category to code mapping of each categorical metadata label, keyed by the label name, such as
   `"task_name/metadata/color"`.

##### get_classes

```python
def get_classes(self) -> dict[str, dict[str, int]]:
```

Return the class names and the class IDs of each task.

The mapping comes from the `LuxonisLoader`. With `filter_task_names`, it has only those tasks. It has the class order from
`class_order_per_task`.

The `LuxonisLoader` adds a `"background"` class with the ID `0` to a segmentation task when all of these are true:

 * A sample of the view has masks of the task, and the masks do not cover the whole image.
 * The task has more than one class.
 * The task has no `"background"` class.

The IDs of the other classes then increase by `1`. Thus the loaders of two views can return different mappings.

Returns

 * `dict[str, dict[str, int]]`: The class name to class ID mapping of each task, keyed by the task name.

##### get_metadata_types

```python
def get_metadata_types(self) -> dict[str, type[int] | type[Category] | type[float] | type[str]]:
```

Return the Python type of each metadata label.

The dataset stores the name of the type. The method maps `"float"`, `"int"`, and `"str"` to the type of that name. It maps
`"Category"` to `int`, because the labels of the loader hold the integer code of a category.

Returns

 * `dict[str, type[int] | type[Category] | type[float] | type[str]]`: The type of each metadata label, keyed by the label name,
   such as `"task_name/metadata/color"`.

##### get_n_keypoints

```python
def get_n_keypoints(self) -> dict[str, int]:
```

Return the number of keypoints of each task.

The count is the number of keypoint names in the skeleton of the task, from the dataset metadata. A task without a skeleton is not
in the result.

Returns

 * `dict[str, int]`: The number of keypoints, keyed by the task name.

#### Attributes

##### dataset

##### input_shapes

The shape `[C, H, W]` of the input image, keyed by `image_source`.

The dictionary has only the `image_source` key, also for a dataset with more than one image source. Each access loads the first
sample, with the augmentations.

##### kpts_mapping_per_task

##### loader