## Introduction

Let's dive into the basics using an example. We'll create a simple application that runs an object detection neural network and
streams color video with visualized neural network detections. We'll use the DepthAI Python API to create the application.

## Creating a pipeline

### Sources

Now, first node we will add is a ColorCamera. This node will automatically select the center camera (which in most devices is the
color camera) and will provide the video stream to the next node in the pipeline. We will use the preview output, resized to
300x300 to fit the mobilenet-ssd input size (which we will define later). For more information about the ColorCamera node, please
refer to the [ColorCamera documentation](https://docs.luxonis.com/software/depthai-components/nodes/color_camera.md)

```python
# First, we want the Color camera as the output
cam_rgb = pipeline.createColorCamera()
cam_rgb.setPreviewSize(300, 300)  # 300x300 will be the preview frame size, available as 'preview' output of the node
cam_rgb.setInterleaved(False)
```

### Detecton network

Up next, let’s define a MobileNetDetectionNetwork node with mobilenet-ssd network. Each neural network we want to use must first
be compiled to a blob file, which is a binary file containing the network weights and configuration and can be loaded to the
device. More info about compiling the network can be found in the [Compiling a neural
network](https://docs.luxonis.com/software/ai-inference/conversion.md) section. The blob file for this example will be compiled
and downloaded automatically using [blobconverter tool](https://github.com/luxonis/blobconverter/tree/master/cli).
`blobconverter.from_zoo()` function returns `Path` to the model, so we can directly put it inside the `detection_nn.setBlobPath()`
function. With this node, the output from nn will be parsed on device side and we’ll receive a ready to use detection objects. For
this to work properly, we need also to set the confidence threshold to filter out the incorrect results.

```python
detection_nn = pipeline.createMobileNetDetectionNetwork()
# Blob is the Neural Network file, compiled for MyriadX. It contains both the definition and weights of the model
# We're using a blobconverter tool to retreive the MobileNetSSD blob automatically from OpenVINO Model Zoo
detection_nn.setBlobPath(blobconverter.from_zoo(name='mobilenet-ssd', shaves=6))
# Next, we filter out the detections that are below a confidence threshold. Confidence can be anywhere between <0..1>
detection_nn.setConfidenceThreshold(0.5)
```

### Sinks

If we wish to see the output of the neural network, we need to add a XLinkOut node. This node will send the data to the host,
where we can process it further. Each XLinkOut node should only be connected to one node, so we need to add one for the detection
network and one for the RGB stream.

```python
# XLinkOut is a "way out" from the device. Any data you want to transfer to host need to be send via XLink
xout_rgb = pipeline.createXLinkOut()
xout_rgb.setStreamName("rgb")

xout_nn = pipeline.createXLinkOut()
xout_nn.setStreamName("nn")
```

### Linking nodes together

Now we need to connect the nodes. We will connect the ColorCamera to the MobileNetDetectionNetwork and the
MobileNetDetectionNetwork to the XLinkOut. We will also connect the ColorCamera to the XLinkOut to get the video stream. Note that
a node output can be connected to multiple nodes inputs at the same time.

```python
cam_rgb.preview.link(xout_rgb.input)
cam_rgb.preview.link(detection_nn.input)
detection_nn.out.link(xout_nn.input)
```

## Uploading and running the pipeline

### Connect to device

In order to upload the constructed pipeline to the device, we must first initialize it. The general practice is to initialize it
using context manager, so we don't have to worry about closing the device after we are done.

The `depthai.Device(pipeline)` call will create a Device object and pass it the created Pipeline object. In the background this
call will check both USB as well as NETWORK interfaces to check if a device is available and ready to accept connections.

```python
with depthai.Device(pipeline) as device:
```

### Initialize queues

To consume the device results, we get two output queues from the device, with stream names we assigned earlier.

```python
    # To consume the device results, we get two output queues from the device, with stream names we assigned earlier
    q_rgb = device.getOutputQueue("rgb")
    q_nn = device.getOutputQueue("nn")
```

### Consuming device messages

We create a infinite loop to make sure the device stays open. In each iteration of the loop, we try to retrieve messages sent by
the device to the queue we created above. `.tryGet()` method will return either the data packet or None if there isn't any.
`.try()` method is similar but will block until a packet is received.

 * If the packet from RGB camera is present, we're retrieving the frame in OpenCV format using getCvFrame: `.getCvFrame()`.
 * When data from nn is received, we take the detections array that contains mobilenet-ssd results: `.detections`.

```python
    while True:
        # we try to fetch the data from nn/rgb queues. tryGet will return either the data packet or None if there isn't any
        in_rgb = q_rgb.tryGet()
        in_nn = q_nn.tryGet()

        if in_rgb is not None:
            # If the packet from RGB camera is present, we're retrieving the frame in OpenCV format using getCvFrame
            frame = in_rgb.getCvFrame()

        if in_nn is not None:
            # when data from nn is received, we take the detections array that contains mobilenet-ssd results
            detections = in_nn.detections
```

### Drawing frames

If we want to view the results, we can do so with OpenCV. We use the `frame` we have received on the rgb queue and draw a
rectangle over the image for every detection consumed on the nn queue. The bbox coordinates we receive are usually normalized in
range [0, 1] so we need to denormalize them first if we wish to draw over our RGB frame. This is done with custom function:
`frameNorm()`.

```python
        if frame is not None:
            for detection in detections:
                # for each bounding box, we first normalize it to match the frame size
                bbox = frameNorm(frame, (detection.xmin, detection.ymin, detection.xmax, detection.ymax))
                # and then draw a rectangle on the frame to show the actual result
                cv2.rectangle(frame, (bbox[0], bbox[1]), (bbox[2], bbox[3]), (255, 0, 0), 2)
            # After all the drawing is finished, we show the frame on the screen
            cv2.imshow("preview", frame)

        # at any time, you can press "q" and exit the main loop, therefore exiting the program itself
        if cv2.waitKey(1) == ord('q'):
            break
```

## Full code

### app.py

```python
# first, import all necessary modules
from pathlib import Path

import blobconverter
import cv2
import depthai
import numpy as np

pipeline = depthai.Pipeline()

# First, we want the Color camera as the output
cam_rgb = pipeline.createColorCamera()
cam_rgb.setPreviewSize(300, 300)  # 300x300 will be the preview frame size, available as 'preview' output of the node
cam_rgb.setInterleaved(False)

detection_nn = pipeline.createMobileNetDetectionNetwork()
# Blob is the Neural Network file, compiled for MyriadX. It contains both the definition and weights of the model
# We're using a blobconverter tool to retreive the MobileNetSSD blob automatically from OpenVINO Model Zoo
detection_nn.setBlobPath(blobconverter.from_zoo(name='mobilenet-ssd', shaves=6))
# Next, we filter out the detections that are below a confidence threshold. Confidence can be anywhere between <0..1>
detection_nn.setConfidenceThreshold(0.5)

# XLinkOut is a "way out" from the device. Any data you want to transfer to host need to be send via XLink
xout_rgb = pipeline.createXLinkOut()
xout_rgb.setStreamName("rgb")

xout_nn = pipeline.createXLinkOut()
xout_nn.setStreamName("nn")

cam_rgb.preview.link(xout_rgb.input)
cam_rgb.preview.link(detection_nn.input)
detection_nn.out.link(xout_nn.input)

# Pipeline is now finished, and we need to find an available device to run our pipeline
# we are using context manager here that will dispose the device after we stop using it
with depthai.Device(pipeline) as device:
    # From this point, the Device will be in "running" mode and will start sending data via XLink

    # To consume the device results, we get two output queues from the device, with stream names we assigned earlier
    q_rgb = device.getOutputQueue("rgb")
    q_nn = device.getOutputQueue("nn")
    # Here, some of the default values are defined. Frame will be an image from "rgb" stream, detections will contain nn results
    frame = None
    detections = []

    # Since the detections returned by nn have values from <0..1> range, they need to be multiplied by frame width/height to
    # receive the actual position of the bounding box on the image
    def frameNorm(frame, bbox):
        normVals = np.full(len(bbox), frame.shape[0])
        normVals[::2] = frame.shape[1]
        return (np.clip(np.array(bbox), 0, 1) * normVals).astype(int)

    while True:
        # we try to fetch the data from nn/rgb queues. tryGet will return either the data packet or None if there isn't any
        in_rgb = q_rgb.tryGet()
        in_nn = q_nn.tryGet()

        if in_rgb is not None:
            # If the packet from RGB camera is present, we're retrieving the frame in OpenCV format using getCvFrame
            frame = in_rgb.getCvFrame()

        if in_nn is not None:
            # when data from nn is received, we take the detections array that contains mobilenet-ssd results
            detections = in_nn.detections

        if frame is not None:
            for detection in detections:
                # for each bounding box, we first normalize it to match the frame size
                bbox = frameNorm(frame, (detection.xmin, detection.ymin, detection.xmax, detection.ymax))
                # and then draw a rectangle on the frame to show the actual result
                cv2.rectangle(frame, (bbox[0], bbox[1]), (bbox[2], bbox[3]), (255, 0, 0), 2)
            # After all the drawing is finished, we show the frame on the screen
            cv2.imshow("preview", frame)

        # at any time, you can press "q" and exit the main loop, therefore exiting the program itself
        if cv2.waitKey(1) == ord('q'):
            break
```