LuxonisParser

Overview

The LuxonisParser offers a simple API for creating datasets from several common dataset formats (all of which are also supported by Roboflow):

COCO JSON format in RoboFlow and FiftyOne variant,
Pascal VOC XML,
Darknet,
YOLOv4 PyTorch,
MT YOLOv6,
CreateML,
TensorFlow CSV,
Classification Directory (a directory with subdirectories for each class), and

Plain Text

1dataset_dir/
2├── train/
3│   ├── class1/
4│   │   ├── img1.jpg
5│   │   ├── img2.jpg
6│   │   └── ...
7│   ├── class2/
8│   └── ...
9├── valid/
10└── test/

Segmentation Mask Directory (a directory with images and corresponding masks).

Plain Text

1dataset_dir/
2├── train/
3│   ├── img1.jpg
4│   ├── img1_mask.png
5│   ├── ...
6│   └── _classes.csv
7├── valid/
8└── test/

The masks must be stored as grayscale PNG images where each pixel value corresponds to a class. The mapping from pixel values to class is defined in the _classes.csv file.

Csv

1Pixel Value, Class
20, background
31, class1
42, class2
53, class3

Dataset Parsing

Parsing starts by initializing the LuxonisParser object with the path to dataset directory. Optionally, you can specify the name and the type (i.e. the format) of the dataset (by default, the name is set to the name of the provided dataset directory, and the type is infered based on dataset directory structure) The dataset directory can either be a path to a local directory or an URL to a directory stored on one of the supported cloud storage providers (the dataset is automatically downloaded in that case). You can also provide the dataset directory as a .zip file.

Python

1from luxonisml.data import LuxonisParser
2from luxonis_ml.enums import DatasetType
3
4dataset_dir = "local/path/to/dataset or URL"
5
6parser = LuxonisParser(
7  dataset_dir: str = dataset_dir,
8  dataset_name: Optional[str] = ... # e.g. "my_dataset",
9  dataset_type: Optional[str] = ... # e.g. DatasetType.COCO
10)

After initializing the LuxonisParser object, parsing can be ran by calling .parse() method on it:

Python

1dataset = parser.parse()

This creates a LuxonisDataset instance containing the data from the provided dataset, keeping the original splits.

CLI Reference

The parsing functionality can be invoked by using the luxonis_ml data parse command.

Command Line

1luxonis_ml data parse path/to/dataset --name my_dataset --type coco

For more detailed information, run luxonis_ml data parse --help.