# base_parser

Python API: `luxonis_ml.data.parsers.base_parser`

## Classes

| Name | Description |
| --- | --- |
|
[`BaseParser`](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/parsers/base_parser.md)
| Base class for dataset-format parsers. |

### class `BaseParser`

```python
class BaseParser(ABC)
```

Base class for dataset-format parsers.

Methods

| Name | Description |
| --- | --- |
|
[`__init__`](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/parsers/base_parser.md)
| Create a parser for a target dataset. |
|
[`discover_dir_splits`](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/parsers/base_parser.md)
| Return present and valid split directories keyed by their canonical split names. |
|
[`from_dir`](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/parsers/base_parser.md)
| Parse all data in a source dataset directory. |
|
[`from_split`](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/parsers/base_parser.md)
| Parse data from one split subdirectory. |
|
[`get_parser_issue_messages`](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/parsers/base_parser.md)
| Return parser issue messages collected during the last parse. |
|
[`parse_dir`](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/parsers/base_parser.md)
| Parse an entire dataset directory into the target dataset. |
|
[`parse_split`](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/parsers/base_parser.md)
| Parse one split subdirectory into the target dataset. |
|
[`reset_parser_issue_messages`](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/parsers/base_parser.md)
| Clear collected parser issue messages. |
|
[`validate`](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/parsers/base_parser.md)
| Validate whether the dataset directory has the expected format. |
|
[`validate_split`](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/parsers/base_parser.md)
| Validate whether a split directory has the expected format. |

#### method `__init__`

```python
def __init__(
    dataset: BaseDataset,
    dataset_type: DatasetType,
    task_name: str | dict[str, str] | None,
    full_warnings: bool = False,
)
```

Create a parser for a target dataset.

Args

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `dataset` | `BaseDataset` | Dataset to populate with parsed records. | required |
| `dataset_type` | `DatasetType` | Source dataset format. | required |
| `task_name` | `str | dict[str, str] | None` | Optional task naming rule. A string is used for all records. A mapping uses class
names as keys and task names as values. | required |
| `full_warnings` | `bool` | Whether to log every skipped-annotation warning. When `False`, only the first 50 warnings are logged
and the rest are summarized. | `False` |

#### class method `discover_dir_splits`

```python
def discover_dir_splits(dataset_dir: Path) -> dict[str, dict[str, Any]]
```

Return present and valid split directories keyed by their canonical split names.

#### method `from_dir`

```python
def from_dir(dataset_dir: Path, **kwargs) -> tuple[list[Path], list[Path], list[Path]]
```

Parse all data in a source dataset directory.

Args

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `dataset_dir` | `Path` | Source dataset directory. | required |
| `**kwargs` |  | Additional parser-specific arguments. |  |

Returns

| Type | Description |
| --- | --- |
| `tuple[list[Path], list[Path], list[Path]]` | Added images for the train, validation, and test splits. |

#### method `from_split`

```python
def from_split(**kwargs) -> ParserOutput
```

Parse data from one split subdirectory.

> **Example**
> ```python
split_kwargs = parser.validate_split(split_path)
if split_kwargs is not None:
    parser.from_split(**split_kwargs)
```

Args

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `**kwargs` |  | Parser-specific arguments, usually produced by `validate_split`. |  |

Returns

| Type | Description |
| --- | --- |
| `ParserOutput` | LDF generator, skeleton metadata, and added images. |

#### method `get_parser_issue_messages`

```python
def get_parser_issue_messages() -> list[ParserIssueMessage]
```

Return parser issue messages collected during the last parse.

#### method `parse_dir`

```python
def parse_dir(dataset_dir: Path, **kwargs) -> BaseDataset
```

Parse an entire dataset directory into the target dataset.

Args

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `dataset_dir` | `Path` | Source dataset directory. | required |
| `**kwargs` |  | Parser-specific arguments. |  |

Returns

| Type | Description |
| --- | --- |
| `BaseDataset` | Dataset with parsed images and annotations. |

Raises

| Type | Description |
| --- | --- |
| `ValueError` | If a parser that expects top-level splits cannot find a `train` directory. |

#### method `parse_split`

```python
def parse_split(
    split: str | None = None,
    random_split: bool = False,
    split_ratios: dict[str, float | int] | None = None,
    **kwargs,
) -> BaseDataset
```

Parse one split subdirectory into the target dataset.

Args

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `split` | `str | None` | Optional split name to assign to parsed data. When set, `split_ratios` and `random_split` are ignored. | `None` |
| `random_split` | `bool` | Whether to generate random splits using `split_ratios`. | `False` |
| `split_ratios` | `dict[str, float | int] | None` | Optional ratios or counts. Float values are treated as ratios; integer values are treated as counts. | `None` |
| `**kwargs` |  | Parser-specific arguments. |  |

Returns

| Type | Description |
| --- | --- |
| `BaseDataset` | Dataset with parsed images and annotations. |

#### method `reset_parser_issue_messages`

```python
def reset_parser_issue_messages()
```

Clear collected parser issue messages.

#### class method `validate`

```python
def validate(dataset_dir: Path) -> bool
```

Validate whether the dataset directory has the expected format.

Args

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `dataset_dir` | `Path` | Source dataset directory. | required |

Returns

| Type | Description |
| --- | --- |
| `bool` | Whether the dataset is in the expected format. |

#### static method `validate_split`

```python
def validate_split(split_path: Path) -> dict[str, Any] | None
```

Validate whether a split directory has the expected format.

Args

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `split_path` | `Path` | Path to a split directory. | required |

Returns

| Type | Description |
| --- | --- |
| `dict[str, Any] | None` | Keyword arguments for `from_split`, or `None` if the split is not in the expected format. |

## Module Attributes

| Name | Description |
| --- | --- |
| [`ParserOutput`](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/parsers/base_parser.md) |  |

### type alias `ParserOutput`

```python
ParserOutput = tuple[DatasetIterator, dict[str, dict], list[Path]]
```
