# LuxonisDataset

## Overview

The `LuxonisDataset` class offers a simple API for creating and managing data in the Luxonis Data Format (LDF). It acts as an
abstraction layer and provides methods for dataset:

 * initialization,
 * ingestion,
 * splitting,
 * merging, and
 * export, cloud synchronization, and deletion.

The following sections will guide you through creation of a LDF dataset. We have prepared a simple toy dataset you can use to
follow along the examples ([ParkingLot.zip](https://drive.google.com/uc?export=download&id=1OAuLlL_4wRSzZ33BuxM6Uw2QYYgv_19N)). It
consists of images of cars and motorcycles on a parking lot, each annotated with a bounding box, keypoints and segmentation mask.

## Dataset initialization

Dataset creation process starts by initializing the `LuxonisDataset` object:

```python
from luxonis_ml.data.datasets import LuxonisDataset

dataset_name: str = ... # e.g. "parking_lot"
dataset = LuxonisDataset(dataset_name)
```

> **Note**
> Datasets can be stored locally or using one of the supported cloud storage providers, including GCS, S3, and Azure Blob storage. By default, the initialized dataset is stored locally.

> **Note**
> If there already exist a dataset with the provided `dataset_name`, it will be automatically loaded instead of initializing a new one. Therefore, beware to use a unique name for each new dataset or pass `delete_local=True` to the `LuxonisDataset` constructor to overwrite an existing one.

If you need more control over storage behavior, the constructor also accepts `team_id`, `bucket_type`, `bucket_storage`, and
`delete_remote` parameters. These are useful when the dataset should live in shared or remote object storage instead of only on
the local machine.

## Adding Data

After dataset initialization, we can start with data ingestion. We must first define a generator function that yields individual
data instances. Each data instance stores path to an image and a single annotation (e.g. a bounding box). So in case of multiple
annotations per image, multiple data instances must be yielded separately.

We define data instances as a Python dictionary with the following structure:

```python
{
    "file": str,  # path to the image file
    "annotation": Optional[dict]  # single image annotation
}
```

where content of the `annotation` field depends on the task type. The following task types are supported:

 * [Classification](https://github.com/luxonis/luxonis-ml/tree/main/luxonis_ml/data#classification)
 * [Bounding Box](https://github.com/luxonis/luxonis-ml/tree/main/luxonis_ml/data#bounding-box)
 * [Keypoints](https://github.com/luxonis/luxonis-ml/tree/main/luxonis_ml/data#keypoints)
 * [Segmentation Mask](https://github.com/luxonis/luxonis-ml/tree/main/luxonis_ml/data#segmentation-mask)
 * [Instance Segmentation](https://github.com/luxonis/luxonis-ml/tree/main/luxonis_ml/data#instance-segmentation)
 * [Array](https://github.com/luxonis/luxonis-ml/tree/main/luxonis_ml/data#array)
 * [Metadata](https://github.com/luxonis/luxonis-ml/tree/main/luxonis_ml/data#metadata)

Below we provide an example generator function for the parking lot dataset, yielding the data instances for bounding box
annotations.

```python
import json
from pathlib import Path

# path to the dataset, replace it with the actual path on your system
dataset_root = Path("data/parking_lot")

def generator():
    for annotation_dir in dataset_root.iterdir():
        with open(annotation_dir / "annotations.json") as f:
            data = json.load(f)

        # get the width and height of the image
        W = data["dimensions"]["width"]
        H = data["dimensions"]["height"]

        image_path = annotation_dir / data["filename"]

        for instance_id, bbox in data["BoundingBoxAnnotation"].items():

            # get unnormalized bounding box coordinates
            x, y = bbox["origin"]
            w, h = bbox["dimension"]

            # get the class name of the bounding box
            class_ = bbox["labelName"]
            yield {
                "file": image_path,
                "annotation": {
                    "class": class_,
                    # normalized bounding box
                    "boundingbox": {
                        "x": x / W,
                        "y": y / H,
                        "w": w / W,
                        "h": h / H,
                    },
                },
            }
```

The generator is then passed to the `add` method of the dataset.

```python
dataset.add(generator())
```

> **Note**
> The `add` method accepts any iterable, not only generators.

## Metadata and sources

Luxonis datasets can store more than just images and labels. Dataset metadata keeps track of:

 * task definitions and class mappings
 * keypoint skeletons
 * categorical metadata encodings
 * dataset source structure

The source structure is represented by `LuxonisSource` and `LuxonisComponent`, which lets one dataset describe multi-component or
multi-sensor inputs as well. For example, one source can contain multiple image components instead of only a single RGB image.

Useful metadata-related methods include:

 * `set_tasks(...)` to define task groups explicitly
 * `set_classes(...)` to register class mappings
 * `get_source_names()` to inspect the available dataset sources
 * `update_source(...)` to update source/component metadata

## Defining Splits

After adding data to the dataset, we can define its splits. There are no restrictions on the split names but in most cases one
should stick to `train`, `val`, and `test` sets. The splits are defined by calling the `make_splits` method on the
`LuxonisDataset` object and passing the desired split ratios in its arguments (by default, the data are split with the 80:10:10
ratio between `train`, `val`, and `test` sets).

```python
dataset.make_splits({
  "train": 0.7,
  "val": 0.2,
  "test": 0.1,
})
```

For a more refined control over the splits, you can pass a dictionary with the split names as keys and lists of file names as
values:

```python

dataset.make_splits({
  "train": ["file1.jpg", "file2.jpg", ...],
  "val": ["file3.jpg", "file4.jpg", ...],
  "test": ["file5.jpg", "file6.jpg", ...],
})
```

Once splits are made, calling the `make_splits` method again will raise an error. If you wish to redefine them, pass
`redefine_splits=True` to the method call.

## Cloud sync and dataset discovery

For remote workflows, `LuxonisDataset` can also:

 * list datasets with `LuxonisDataset.list_datasets(...)`
 * pull missing or all media locally with `pull_from_cloud(...)`
 * push local data to remote object storage with `push_to_cloud(...)`

These methods are especially useful when the same dataset is shared across training machines or team environments.

## Dataset Cloning

You can clone an existing dataset to create a copy with a new name. This is useful for testing changes without affecting the
original dataset. Cloning is done by calling the `clone` method on the `LuxonisDataset` object and passing the desired name of the
new dataset.

```python
dataset_clone = dataset.clone(new_dataset_name="dataset_clone")
```

## Dataset Merging

Datasets can also be merged together. This is beneficial for combining multiple datasets into a larger, unified dataset for
comprehensive training or analysis. Merging is done by calling the `merge_with` method on the first `LuxonisDataset` object and
passing the second one as an argument. You can choose between two different merging modes:

 * `inplace`: the first dataset is modified to include data from the second dataset
 * `out-of-place`: a new dataset is created from the combination of two existing datasets

```python
# inplace merging
dataset1.merge_with(dataset2, inplace=True)
# OR out-of-place merging
dataset_merge = dataset1.merge_with(dataset2, inplace=False, new_dataset_name="dataset_merge")
```

## Dataset export

`LuxonisDataset` can export data back out of LDF into common dataset formats. This is useful when you want to prepare data in
LuxonisML, but train or inspect it in another toolchain.

```python
from luxonis_ml.enums import DatasetType

dataset.export("exports/coco", dataset_type=DatasetType.COCO)
```

The current exporters cover native LDF export as well as several standard formats, including COCO, Pascal VOC, Darknet, YOLOv4,
YOLOv6, YOLOv8 task-specific exporters, TensorFlow CSV, CreateML, FiftyOne Classification, Classification Directory, and
Segmentation Mask Directory.

## CLI Reference

The `luxonis_ml` CLI provides a set of various useful commands for managing datasets. These commands are accessible via the
`luxonis_ml data` command.

The available commands are:

 * `luxonis_ml data ls` - lists all datasets
 * `luxonis_ml data info <dataset_name>` - prints information about the dataset
 * `luxonis_ml data inspect <dataset_name>` - renders the data in the dataset on screen using `cv2`
 * `luxonis_ml data delete <dataset_name>` - deletes the dataset

For more information, run `luxonis_ml data --help` or pass the `--help` flag to any of the above commands.