# parquet

Python API: `luxonis_ml.data.utils.parquet`

## Classes

### ParquetFileManager

Manage append-style writes across partitioned parquet files.

Rows are buffered in memory and flushed to the current parquet file. A new file is selected every `num_rows` writes. Filenames are
zero-padded numeric counters such as `0000000000.parquet`.

#### Methods

##### init

```python
def __init__(directory: PathType, num_rows: int = 100000):
```

Manage writing rows into partitioned parquet files.

> **Example**
> ```pycon
>>> record = {
...     "file": "image.jpg",
...     "source_name": "image",
...     "task_name": "detection",
...     "class_name": "car",
...     "instance_id": 0,
...     "task_type": "boundingbox",
...     "annotation": "{}",
... }
>>> manager = ParquetFileManager("/tmp/ldf-parquet-example", num_rows=2)
>>> manager.num_rows
2
```

Parameters

 * `directory` (`PathType`): Local directory where parquet files are stored.
 * `num_rows` (`int`): Maximum rows per parquet file before a new file is created.

##### remove_duplicate_uuids

```python
def remove_duplicate_uuids(overwrite_uuids: set[str]):
```

##### write

```python
def write(uuid: str, data: ParquetRecord, group_id: str):
```

Write a row to the current parquet file.

Parameters

 * `uuid` (`str`): Unique row identifier.
 * `data` (`ParquetRecord`): Annotation row data.
 * `group_id` (`str`): Unique identifier for the sample group.

#### Attributes

##### buffer

In-memory column buffer for the current file.

##### current_file

Path to the current parquet file.

##### dir

Directory containing parquet files.

##### num

Current parquet file index.

##### num_rows

Maximum rows written to one parquet file.

##### parquet_files

Existing parquet files discovered in `dir`.

##### row_count

Number of rows currently buffered or loaded from the current file.

### ParquetRecord

Single annotation row written to parquet.

#### Attributes

##### annotation

Optional serialized annotation JSON.

##### class_name

Optional class name.

##### file

Image or source file path.

##### instance_id

Optional instance identifier.

##### sample_metadata

Optional serialized metadata JSON for the entire sample.

##### source_name

Source component name.

##### task_name

Task name.

##### task_type

Optional task type.

## Attributes

### DEFAULT_METADATA