# Create, upload, and export datasets

## Overview

Create a dataset by defining its tasks, then populate it from an annotated archive, individual images, Snaps, or a recurring data
ingestion. Hub imports data in a background job and can later export a compatible archive for use outside Hub.

> **Note**
> Before importing annotated data, identify its task types, task names, and classes. Hub drops annotations that do not match the dataset definition.

## Create a dataset and upload data

The following example creates an object-detection dataset for pallet images.

Open [Luxonis Hub](https://hub.luxonis.com/ai/datasets), select Datasets, and click Add Dataset. Enter a descriptive name, such as
`Warehouse Pallets`.

Choose the Object Detection task preset, enter a task name, add the `pallet` class, and click Add.

Alternatively, select a Project to inherit its task names and classes. Use Custom when the dataset contains several tasks or needs
a task definition that is not represented by a preset.

Choose an upload source, or close the dialog to keep the new dataset empty:

 * Zip of existing data: imports images and supported annotations from an archive in a supported dataset format. See
   [LuxonisParser](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-parser.md) for more
   details on these supported formats.
 * Images: uploads image files and lets you choose their split and tags.
 * Cancel: closes the upload dialog and leaves the new dataset empty so you can add data later.

Select the archive or images and click Upload. Hub queues an upload job and changes the dataset to Ready when processing succeeds.
Open the notification or the Jobs page to inspect saved and dropped annotation counts or troubleshoot a failed import.

Use Add Data from an existing dataset to upload more archives or images later.

### Archive parsing reference

Hub detects a supported format from the extracted directory structure. Supported import layouts include:

| Data type | Auto-detected layouts |
| --- | --- |
| Object Detection | COCO, Pascal VOC, Darknet, YOLOv4, YOLOv6, YOLOv8, Create ML, TensorFlow CSV, Ultralytics NDJSON |
| Classification | Classification directory, FiftyOne classification, Luxonis native format |
| Semantic Segmentation | Segmentation-mask directory, Unity SOLO, Luxonis native format |
| Instance Segmentation | COCO, YOLOv8 instance segmentation, Ultralytics NDJSON instance segmentation, Unity SOLO, Luxonis native
format |
| Keypoint Detection | COCO, YOLOv8 keypoints, Ultralytics NDJSON keypoints, Unity SOLO, Luxonis native format |
| OCR and mixed tasks | Luxonis native format |

For layout examples and the canonical format names, see
[LuxonisParser](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-parser.md).

> **Prepare the archive root carefully**
> Hub decides whether to parse annotations when an archive contains common annotation files such as JSON, YAML, XML, TXT, CSV, LABELS, or NDJSON. Place the dataset layout at the archive root. An extra enclosing directory can prevent format detection and produce a “zero samples parsed” error.

> **Splits, statuses, and unmatched classes**
> If the source format has no split information, Hub assigns samples using an 80/10/10 train, validation, and test distribution. Regular uploads enter the annotation workflow as Completed. Annotations with classes or types not defined on the dataset are dropped; review the job log for the count.

When uploading image files without annotations, choose their split and optional tags in the upload form. These samples are ready
to annotate after processing.

## Add a single Snap

Use this path to move a useful image from device operations into a training dataset.

In Hub, open Control → Snaps, then select the Snap you want to keep. See
[Snaps](https://docs.luxonis.com/cloud/features/event-storage/snaps.md) for capture and storage details.

Click Add to Dataset, choose the destination dataset and split, then confirm. Hub queues the sample for ingestion.

## Configure a data ingestion

A Data Ingestion collects matching Snaps into a dataset on a schedule or over a one-time range. Use it to turn production
observations into a repeatable data-collection pipeline.

Open Datasets, select the Data Ingestions tab, and click Add Ingestion.

Enter a name, select the destination dataset, and optionally attach tags to imported samples. Choose All to include every matching
Snap or Random and a ratio to sample a percentage.

Choose a one-time date range or a recurring frequency. Optionally select a source device or application to restrict which Snaps
are collected; leave Source Device or App as None to collect from every source that matches the other settings. Save the ingestion
and use Run when you want to start it immediately.

Use the Ingestion Runs table to monitor each execution and open its background job when more detail is needed.

If a Snap contains detections whose label names match an object-detection task, Hub imports them as Pre-annotated boxes. Review
those annotations before changing the samples to Completed.

## Export a dataset

Open the dataset action menu, select Export, and choose one of the formats compatible with the dataset's tasks. Selecting the
format queues the export job immediately.

Open Jobs, find the completed Dataset Export job, and click Download Export. Download links are temporary, so start another export
if the link has expired.

### Export format reference

Hub displays only formats that can represent every task in the dataset. The Luxonis native format is the fallback for mixed or
unsupported combinations.

| Dataset task | Compatible export families |
| --- | --- |
| Object Detection | COCO, Pascal VOC, Darknet, YOLOv4, YOLOv6, YOLOv8, Create ML, TensorFlow CSV, Ultralytics NDJSON, Luxonis |
| Classification | Classification directory, FiftyOne classification, Luxonis |
| Semantic Segmentation | Segmentation-mask directory, Unity SOLO, Luxonis |
| Instance Segmentation | COCO, YOLOv8 instance segmentation, Ultralytics NDJSON instance segmentation, Unity SOLO, Luxonis |
| Keypoint Detection | COCO, YOLOv8 keypoints, Ultralytics NDJSON keypoints, Unity SOLO, Luxonis |
| OCR | Luxonis |

For a dataset containing multiple tasks, the available list is the intersection of the formats that support all required
annotation types.