# DepthAI Nodes

## Overview

In DepthAI pipelines, messages are used to send information between nodes. While DepthAI natively supports many message and node
types, some use cases need extra host-side components for postprocessing, filtering, visualization, or higher-level pipeline
composition.

depthai-nodes is an open-source Python library of host-side nodes and message types for DepthAI v3 pipelines. It extends standard
DepthAI workflows with parser, utility, and helper nodes that run on the host machine while remaining compatible with the rest of
the pipeline.

The library is organized into two main modules:

 * node for host-side pipeline components
 * message for richer message types used by those components

For more examples and package-level documentation, see the [GitHub repository](https://github.com/luxonis/depthai-nodes) and the
[API reference](https://docs.luxonis.com/software-v3/ai-inference/inference/depthai-nodes/api-reference.md).

## Installation

depthai-nodes requires Python 3.10 or newer and is intended for depthai v3 pipelines.

Install it from PyPI with:

```bash
pip install depthai-nodes
```

If you want to work from source, clone the repository and install it locally:

```bash
git clone git@github.com:luxonis/depthai-nodes.git
cd depthai-nodes
pip install .
```

## Modules

### Node

The node module provides a collection of custom node objects that extend the functionality of standard DepthAI nodes. These nodes
are grouped into parser, utility, and helper categories.

#### Typical workflow

For most AI use cases, ParsingNeuralNetwork is the main entry point. It creates the underlying NeuralNetwork node together with
the parser nodes required for the selected model. Model sources can be provided as a Luxonis Models reference, a local [NN
Archive](https://docs.luxonis.com/software-v3/ai-inference/nn-archive.md), or a dai.NNModelDescription.

Related helpers include:

 * HostParsingNeuralNetwork for a host-side version of the same workflow
 * ParserGenerator when you need to generate parser nodes from an NN Archive

#### Parsers

Parser nodes are used to interpret the outputs of AI models. Their primary purpose is to encapsulate and abstract away
postprocessing logic, allowing users to interact directly with clean, structured outputs. Depending on the parser, outputs can be
standard DepthAI messages such as dai.ImgDetections, dai.TrackedFeatures, or dai.ImgFrame, or custom depthai_nodes.message types.

Below is an overview of the available parser nodes, along with their corresponding tasks and usage examples. We list only a small
fraction of examples as many more are available on [Luxonis Models](https://models.luxonis.com/). For detailed information on each
individual parser, please refer to the [Parser Node
documentation](https://github.com/luxonis/depthai-nodes/tree/main/depthai_nodes/node#parser-nodes).

| Parser | Task | Usage Example |
| --- | --- | --- |
| `YOLOExtendedParser` | Object Detection | [YOLOv8 Instance Segmentation
Nano](https://models.luxonis.com/luxonis/yolov8-instance-segmentation-nano/9c1ea8c4-7ab4-46d2-954b-de237c7b4a05), [YOLOv8 Nano
Pose Estimation](https://models.luxonis.com/luxonis/yolov8-nano-pose-estimation/12acd8d7-25c0-4a07-9dff-ab8c5fcae7b1) |
| `YuNetParser` | Object Detection | [YuNet](https://models.luxonis.com/luxonis/yunet/5d635f3c-45c0-41d2-8800-7ca3681b1915) |
| `SCRFDParser` | Object Detection | [SCRFD Person
detection](https://models.luxonis.com/luxonis/scrfd-person-detection/c3830468-3178-4de6-bc09-0543bbe28b1c) |
| `MPPalmDetectionParser` | Object Detection | [MediaPipe Palm
Detection](https://models.luxonis.com/luxonis/mediapipe-palm-detection/9531aba9-ef45-4ad3-ae03-808387d61bf3) |
| `PPTextDetectionParser` | Object Detection | [Paddle Text
Detection](https://models.luxonis.com/luxonis/paddle-text-detection/131d855c-60b1-4634-a14d-1269bb35dcd2) |
| `ClassificationParser` | Classification |
[EfficientNet-Lite](https://models.luxonis.com/luxonis/efficientnet-lite/fdacd30d-97f4-4c55-843f-8b7e872d8acb) |
| `ClassificationSequenceParser` | Classification | [Paddle Text
Recognition](https://models.luxonis.com/luxonis/paddle-text-recognition/9ae12b58-3551-49b1-af22-721ba4bcf269) |
| `SegmentationParser` | Segmentation |
[DeepLab-V3-Plus](https://models.luxonis.com/luxonis/deeplab-v3-plus/1189a661-fd0a-44fd-bc9e-64b94d60cb49) |
| `FastSAMParser` | Segmentation | [FastSAM s](https://models.luxonis.com/luxonis/fastsam-s/4af2416c-2ba4-4c85-97d0-fd26f089fc69)
|
| `KeypointParser` | Keypoints Detection | [MediaPipe Hand
Landmarker](https://models.luxonis.com/luxonis/mediapipe-hand-landmarker/42815cca-deab-4860-b4a9-d44ebbe2988a) |
| `SuperAnimalParser` | Keypoints Detection | [SuperAnimal
Landmarker](https://models.luxonis.com/luxonis/superanimal-landmarker/894cf1a2-23fb-4c96-8944-a0d1be38a7c7) |
| `HRNetParser` | Keypoints Detection |
[Lite-HRNet](https://models.luxonis.com/luxonis/lite-hrnet/c7c9e353-9f6d-43e1-9b45-8edeae82db70) |
| `XFeatMonoParser` | Feature Matching | [XFeat](https://models.luxonis.com/luxonis/xfeat/6c2790a1-bf68-4e89-a4b3-5c9ae68183b5) |
| `XFeatStereoParser` | Feature Matching | [XFeat](https://models.luxonis.com/luxonis/xfeat/6c2790a1-bf68-4e89-a4b3-5c9ae68183b5)
|
| `LaneDetectionParser` | Other | [Ultra Fast Lane
Detection](https://models.luxonis.com/luxonis/ultra-fast-lane-detection/b15d067f-2cde-48a0-85bf-52e1174b1ac0) |
| `MLSDParser` | Other | [M-LSD](https://models.luxonis.com/luxonis/m-lsd/9e3e01d8-2303-4113-bf69-cb10ec56ad5b) |
| `EmbeddingsParser` | Other | [ArcFace](https://models.luxonis.com/luxonis/arcface/e24a577e-e2ff-4e4f-96b7-4afb63155eac) |
| `RegressionParser` | Other | [Head pose
estimation](https://models.luxonis.com/luxonis/head-pose-estimation/068ac18a-de71-4a6e-9f0f-42776c0ef980) |
| `MapOutputParser` | Other | [Depth Anything
V2](https://models.luxonis.com/luxonis/depth-anything-v2/c5bf9763-d29d-4b10-8642-fbd032236383) |
| `ImageOutputParser` | Other | [Zero-DCE](https://models.luxonis.com/luxonis/zero-dce/8eaae754-6195-4766-a39c-2d19a856a492) |

#### Utility & Helpers

This group includes nodes designed to assist with common tasks such as data processing, filtering, and pipeline management. It
also contains abstract base nodes that serve as templates for building custom node implementations.

Below is a brief overview of the available utility and helper nodes, along with their corresponding tasks and usage explanations.
You can find more information about each individual node in the [Utility & Helper Nodes
documentation](https://github.com/luxonis/depthai-nodes/tree/main/depthai_nodes/node#utility--helper-nodes).

| Node | Task | Purpose |
| --- | --- | --- |
| `BaseHostNode` | Abstract Class | Host node base class |
| `BaseThreadedHostNode` | Abstract Class | Threaded host node base class |
| `ApplyColormap` | Image Processing | Applies a colormap |
| `ApplyDepthColormap` | Image Processing | Applies percentile-normalized depth or disparity coloring |
| `DepthMerger` | Image Processing | Merges depth and detections |
| `FrameCropper` | Image Processing | Crops and resizes image regions from detections or upstream manip-config groups |
| `ImgFrameOverlay` | Image Processing | Overlays two images |
| `Tiling` | Image Processing | Produces tiled `dai.ImageManipConfig` groups for downstream cropping pipelines |
| `ExtendedNeuralNetwork` | Neural Network Processing | Convenience wrapper around `dai.node.NeuralNetwork` for model-zoo and NN
archive sources |
| `ParsingNeuralNetwork` | Neural Network Processing | Creates a `NeuralNetwork` node and relevant parser(s) for the given model |
| `HostParsingNeuralNetwork` | Neural Network Processing | Host-side `ParsingNeuralNetwork` implementation |
| `ParserGenerator` | Neural Network Processing | Generates parsers from the given NN archive |
| `CoordinatesMapper` | Detection and Filtering | Remaps coordinates between image-transformation spaces for supported messages |
| `HostSpatialsCalc` | Detection and Filtering | Computes host-side spatial coordinates using depth data and calibration
information |
| `ImgDetectionsFilter` | Detection and Filtering | Filters detections message |
| `InstanceToSemanticMask` | Detection and Filtering | Converts instance-id masks into semantic masks |
| `GatherData` | Data Management | Gathers messages into one |
| `MessageCollector` | Data Management | Collects same-timestamp messages from a stream into a single collection |
| `SnapsUploader` | Cloud Integration | Uploads `SnapData` messages to the Luxonis Hub Events API |

### Message

The message module provides a set of custom message types that extend the standard DepthAI messages. These custom types make it
easier to work with the outputs of various AI models and integrate them into visualization or processing pipelines.

Here is a quick overview of the message types currently implemented in the library. For field-level details, see the [Message Type
documentation](https://github.com/luxonis/depthai-nodes/tree/main/depthai_nodes/message#message-types) and the [API
reference](https://docs.luxonis.com/software-v3/ai-inference/inference/depthai-nodes/api-reference.md).

| Message Type | Stores |
| --- | --- |
| `Classifications` | Object class labels with probability scores |
| `Cluster` | A group of related points |
| `Clusters` | Multiple `Cluster` objects |
| `Collection` | A typed collection of messages or other same-type items |
| `GatheredData` | Reference data together with the gathered messages for that reference |
| `Keypoints` | Keypoints together with optional skeleton edges |
| `Line` | Line coordinates with confidence |
| `Lines` | Multiple `Line` objects |
| `Map2D` | 2D float array |
| `Prediction` | Single float value |
| `Predictions` | Multiple `Prediction` objects |
| `SegmentationMask` | 2D integer array |
| `SnapData` | Single snap event with files, tags, and extra metadata for Luxonis Hub upload |