# Standalone mode

Standalone mode allows [RVC2-based](https://docs.luxonis.com/hardware/platform/rvc/rvc2.md) cameras to start the ([flashed
application](#Standalone%2520mode-Flash%2520the%2520pipeline)) automatically when it gets powered on, without being connected to
any particular host computer. This can especially be useful in multi-cam architectures, where each camera can run its own
application without any dependency on the host computer. Each camera can be either a server or a client, and can communicate with
other cameras or servers via network protocols (eg. HTTP, TCP, UDP, MQTT...).

In contrast, Peripheral mode means that the camera is directly connected to a specific host computer. Host computer connects to an
idle device, uploads pipeline + assets (like NN models), and then communicates with the device (eg. gets video stream).

Compared to Peripheral mode, Standalone mode:

 * Doesn't require a host computer to start the application, and can connect to different computers/servers independently
 * Is more robust to any instabilities (eg. networking issues, where connection between camera and host computer would drop), as
   it will auto-restart the application
 * Is faster to start, as host computer doesn't need to send over the pipeline + assets (takes a few seconds)

## Support

For RVC2-based cameras, Standalone mode supports depends on flash memory and ability to communicate with the outside world:

 * [OAK PoE](https://docs.luxonis.com/hardware/platform/deploy/poe-deployment-guide.md)
   ([RVC2-based](https://docs.luxonis.com/hardware/platform/rvc/rvc2.md)) have on-board flash memory, and can communicate with the
   outside world via network protocols (eg. HTTP, TCP, UDP, MQTT...)
 * [OAK USB](https://docs.luxonis.com/hardware/platform/deploy/usb-deployment-guide.md)
   ([RVC2-based](https://docs.luxonis.com/hardware/platform/rvc/rvc2.md)) - not supported, as they can't communicate with the
   outside world
 * [Deprecated] OAK IoT ([RVC2-based](https://docs.luxonis.com/hardware/platform/rvc/rvc2.md)) have on-board memory and integrated
   ESP32, which was able to communicate with the outside world via WiFi/Bluetooth.

Devices that have integrated Linux computers can setup their own applications to run on boot:

 * [OAK-D CM4](https://shop.luxonis.com/products/oak-d-cm4) and [OAK-D CM4
   PoE](https://shop.luxonis.com/collections/oak-cameras-1/products/oak-d-cm4-poe) have integrated Raspberry Pi Compute Module 4
 * Both [RVC3](https://docs.luxonis.com/hardware/platform/rvc/rvc3.md) ([RAE
   Robot](https://docs.luxonis.com/hardware/rae/get-started.md)) and
   [RVC4](https://docs.luxonis.com/hardware/platform/rvc/rvc4.md)
   ([OAK4](https://docs.luxonis.com/hardware/platform/rvc/rvc4.md#Robotics%20Vision%20Core%204%20(RVC4)-OAK4%20cameras)) chips are
   running Linux OS, so users can directly SSH into the device, and can setup their own applications to run on boot.

## OAK PoE standalone mode

To "communicate" with the outside world (computer), OAK POE cameras can use [Script
node](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/script.md) to send/receive networking packets
(HTTP/TCP/UDP...). Here are a few examples:

 * [TCP streaming](https://github.com/luxonis/oak-examples/tree/master/gen2-poe-tcp-streaming) (camera being either server or
   client)
 * [HTTP server](https://docs.luxonis.com/software/depthai/examples/script_mjpeg_server.md#script-mjpeg-server)
 * [HTTP client](https://docs.luxonis.com/software/depthai/examples/script_http_client.md)
 * [MQTT client](https://github.com/luxonis/oak-examples/tree/master/gen2-poe-mqtt)

> **DNS resolver**
> Standalone mode is missing a DNS resolver, so you will need to use IP addresses instead of domain names.

### Converting to standalone mode

Since there won't be direct communication between the host and the device, you first need to remove all XLinkIn nodes, which is
used in peripheral mode for communication between the host and the device.

We'll need to add Script node to the pipeline, and stream all data we want (eg. encoded video, NN metadata, IMU results, etc.) to
the Script node first, where it will be sent over the network.

#### Example

Let's update [YOLOv8](https://docs.luxonis.com/software-v3/depthai/examples/yolov8_nano.md) example to standalone mode. This
example uses YOLOv8 model to detect objects in the video stream, and sends the video stream + metadata to the host computer, where
it gets visualized (boudning boxes) and displayed to the user.

First, we'll need to separate the code into two parts:

 * One part will be the pipeline definition (which will be flashed to the device)
 * The other part will be the host computer code (receiving data, visualizing and displaying it)

In example's source code (Python), we can see that the pipeline definition code is in rows 41 to 75. We can copy this definition
to a separate file (eg. oak.py). I'll also remove XLinkOut nodes, as they're ignored in standalone mode, and instead create a
Script node, which will be used to send the data over the network.

```python
pipeline = dai.Pipeline()

# Define sources and outputs
camRgb = pipeline.create(dai.node.ColorCamera)
detectionNetwork = pipeline.create(dai.node.YoloDetectionNetwork)
# Properties
camRgb.setPreviewSize(640, 352)
camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
camRgb.setInterleaved(False)
camRgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR)
camRgb.setFps(40)

# Network specific settings
detectionNetwork.setConfidenceThreshold(0.5)
detectionNetwork.setNumClasses(80)
detectionNetwork.setCoordinateSize(4)
detectionNetwork.setIouThreshold(0.5)
detectionNetwork.setBlobPath(nnPath)
detectionNetwork.setNumInferenceThreads(2)
detectionNetwork.input.setBlocking(False)

# Create Script node that will handle TCP communication
script = pipeline.create(dai.node.Script)
script.setProcessor(dai.ProcessorType.LEON_CSS)
script.setScript("""
while True:
    detections = node.io["detection_in"].get().detections
    img = node.io["frame_in"].get()
""")
# Link outputs (RGB stream, NN output) to the Script node
detectionNetwork.passthrough.link(script.inputs['frame_in'])
detectionNetwork.out.link(script.inputs['detection_in'])
```

Now, we only need to focus on the Script node part, so it's spin up TCP server and send frames + detections to connected clients.
We already have [TCP streaming sample
code](https://github.com/luxonis/oak-examples/blob/master/gen2-poe-tcp-streaming/oak.py#L21-L45), so we can use that as a base.
We'll also have to stream detection results (metadata) to the client, so we'll add that code.

Final Script node code will look like this:

```python
import socket
import time
import threading
node.warn("Server up")
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("0.0.0.0", 5000)) # Create TCP server on port 5000
server.listen()

while True:
    conn, client = server.accept()
    node.warn(f"Connected to client IP: {client}")
    try:
        while True:
            detections = node.io["detection_in"].get().detections # Read ImgDetections message, only get detections
            img = node.io["frame_in"].get() # Read ImgFrame message
            node.warn('Received frame + dets')
            img_data = img.getData()
            ts = img.getTimestamp()

            det_arr = []
            for det in detections:
                det_arr.append(f"{det.label};{(det.confidence*100):.1f};{det.xmin:.4f};{det.ymin:.4f};{det.xmax:.4f};{det.ymax:.4f}")
            det_str = "|".join(det_arr) # Serialize detections to string, which will get sent to the client

            header = f"IMG {ts.total_seconds()} {len(img_data)} {len(det_str)}".ljust(32)
            node.warn(f'>{header}<')
            conn.send(bytes(header, encoding='ascii')) # Send over the header
            if 0 < len(det_arr): # Send over serialized detections (if there are any)
                conn.send(bytes(det_str, encoding='ascii'))
            conn.send(img_data) # Send over the actual image frame
    except Exception as e:
        node.warn("Client disconnected")
```

Now that we have Pipeline side with Script node ready, we need to create host.py script that will connect to the camera's TCP
server, receive the video stream + metadata, and visualize + display the data. We can use [host.py
script](https://github.com/luxonis/oak-examples/blob/master/gen2-poe-tcp-streaming/host.py) as a base, and modify it to also
receive and visualize detections:

```python
import socket
import re
import cv2
import numpy as np

# Enter your own IP! After you run oak.py script, it will print the IP in the terminal
OAK_IP = "10.12.101.188"

labels =  [ "person", "bicycle", "car", "motorbike", "aeroplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "sofa", "pottedplant", "bed", "diningtable", "toilet", "tvmonitor", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush" ]

def get_frame(socket, size):
    bytes = socket.recv(4096)
    while True:
        read = 4096
        if size-len(bytes) < read:
            read = size-len(bytes)
        bytes += socket.recv(read)
        if size == len(bytes):
            return bytes

sock = socket.socket()
sock.connect((OAK_IP, 5000))

try:
    COLOR = (127,255,0)
    while True:
        header = str(sock.recv(32), encoding="ascii")
        chunks = re.split(' +', header)
        if chunks[0] == "IMG":
            print(f">{header}<")
            ts = float(chunks[1])
            imgSize = int(chunks[2])
            det_len = int(chunks[3])

            if 0 < det_len: # If there are detections, read them
                det_str = str(sock.recv(det_len), encoding="ascii")

            img = get_frame(sock, imgSize) # Get image frame
            img_planar = np.frombuffer(img, dtype=np.uint8).reshape(3, 352, 640) # Reshape (it's in planar)
            img_interleaved = img_planar.transpose(1, 2, 0).copy() # Convert to interleaved (cv2 requires this)
            # Visualize detections:
            if 0 < det_len:
                dets = det_str.split("|") # Deserialize detections
                for det in dets:
                    det_section = det.split(";")
                    class_id = int(det_section[0])
                    confidence = float(det_section[1])
                    bbox = [ # Convert from relative to absolute
                        int(float(det_section[2]) * img_interleaved.shape[1]),
                        int(float(det_section[3]) * img_interleaved.shape[0]),
                        int(float(det_section[4]) * img_interleaved.shape[1]),
                        int(float(det_section[5]) * img_interleaved.shape[0])
                    ]
                    cv2.putText(img_interleaved, labels[class_id], (bbox[0] + 10, bbox[1] + 20), cv2.FONT_HERSHEY_TRIPLEX, 0.5, COLOR)
                    cv2.putText(img_interleaved, f"{int(confidence)}%", (bbox[0] + 10, bbox[1] + 40), cv2.FONT_HERSHEY_TRIPLEX, 0.5, COLOR)
                    cv2.rectangle(img_interleaved, (bbox[0], bbox[1]), (bbox[2], bbox[3]), COLOR, 2)

            # Display the frame with visualized detections
            cv2.imshow("Img", img_interleaved)

        if cv2.waitKey(1) == ord('q'):
            break
except Exception as e:
    print("Error:", e)

sock.close()
```

Full code (both oak.py and host.py) can be [found
here](https://github.com/luxonis/oak-examples/tree/master/gen2-poe-tcp-streaming/poe-host-yolo). Note that for Standalone mode,
you might want to flash static IP, so you don't have to change the IP in the code every time.

## Bootloader

[Bootloader](https://docs.luxonis.com/software-v3/depthai/depthai-components/bootloader.md) handles booting up the flashed
application (standalone mode), and we suggest using the latest bootloader, which can be flashed using the [Device
Manager](https://docs.luxonis.com/software-v3/depthai/depthai-components/bootloader.md) (GUI tool). To view the API code behind
it, see [Flash Bootloader](https://docs.luxonis.com/software-v3/depthai/examples/flash_bootloader.md) example code (just API
script).

## Flash the pipeline

After you have the (standalone) [Pipeline](https://docs.luxonis.com/software-v3/depthai/depthai-components/pipeline.md) defined
and latest [Bootloader](https://docs.luxonis.com/software-v3/depthai/depthai-components/bootloader.md) flashed on the device, you
can flash the pipeline to the device, along with its assets (eg. NN models). You can flash the pipeline with the following
snippet:

```python
import depthai as dai

pipeline = dai.Pipeline()

# Define standalone pipeline; add nodes and link them
# cam = pipeline.create(dai.node.ColorCamera)
# script = pipeline.create(dai.node.Script)
# ...

# Flash the pipeline
(f, bl) = dai.DeviceBootloader.getFirstAvailableDevice()
bootloader = dai.DeviceBootloader(bl)
progress = lambda p : print(f'Flashing progress: {p*100:.1f}%')
bootloader.flash(progress, pipeline)
```

After successfully flashing the pipeline, it will get started automatically when you power up the device. If you would like to
change the flashed pipeline, simply re-flash it again.

### DepthAI Application Package (.dap)

Alternatively, you can also flash the pipeline with the [Device
Manager](https://docs.luxonis.com/software-v3/depthai/depthai-components/bootloader.md). For this approach, you will need a
Depthai Application Package (.dap), which you can create with the following script:

```python
import depthai as dai

pipeline = dai.Pipeline()

# Define standalone pipeline; add nodes and link them
# cam = pipeline.create(dai.node.ColorCamera)
# script = pipeline.create(dai.node.Script)
# ...

# Create Depthai Application Package (.dap)
(f, bl) = dai.DeviceBootloader.getFirstAvailableDevice()
bootloader = dai.DeviceBootloader(bl)
bootloader.saveDepthaiApplicationPackage(pipeline=pipeline, path=<path_of_new_dap>)
```

## Clear flash

Since pipeline will start when powering the device, this can lead to unnecessary heating. If you would like to clear the flashed
pipeline, use the code snippet below.

```python
import depthai as dai
(f, bl) = dai.DeviceBootloader.getFirstAvailableDevice()
if not f:
    print('No devices found, exiting...')
    exit(-1)

with dai.DeviceBootloader(bl) as bootloader:
    bootloader.flashClear()
    print('Successfully cleared bootloader flash')
```

## Factory reset

In case you have soft-bricked your device, or just want to clear everything (flashed pipeline/assets and bootloader config), we
recommend using the [Device Manager](https://docs.luxonis.com/software-v3/depthai/depthai-components/bootloader.md). Factory reset
will also flash the latest bootloader.