# Stitching

Combines N time-synchronized image streams into a single stitched image. Typical sources are the camera modules of [OAK
FFC](https://docs.luxonis.com/hardware/platform/deploy/ffc.md) devices pointed in different directions, or cameras of several
devices in one pipeline.

> **Note**
> Runs on the host by default. On RVC4 you can offload it to the device with `setRunOnHost(False)`. On RVC2 only host-side processing is supported. Host execution requires DepthAI built with OpenCV support (`DEPTHAI_OPENCV_SUPPORT`, enabled by default). Added in DepthAI v3.10.0, Stitching remains a beta node in v3.11.0 (`dai.beta.node.Stitching` in Python, `dai::beta::node::Stitching` in C++). It is planned to leave beta in v3.12.0 and move to `dai.node.Stitching` / `dai::node::Stitching`. Until then, its API may change between releases.

## Stitching modes

The two modes below use the same inputs: five OAK4 devices arranged in a ring on a city square.

For configuration, go to [Panorama](#Panorama) or [Planar projection](#Planar%20projection). For pipeline setup, start with [How
to place it](#How%20to%20place%20it).

### Panorama example

Panorama stitching provides a view of the surroundings by warping the inputs onto a common sphere, cylinder or plane and blending
them into one panorama. This example shows a full 360° view.

### Planar projection example

Planar projection provides a bird's-eye view of the ground by projecting the inputs onto a plane in the scene and rendering it
from a virtual camera. This example looks down at the pavement around the rig; the cameras do not need to overlap.

The central hole is the ground directly beneath the rig, outside all cameras' views.

## How to place it

Build the node with the camera outputs it should stitch. The number of inputs is fixed at build time. This minimal setup uses the
default image-content panorama registration, which requires overlapping views with enough texture.

#### Python

```python
import depthai as dai

with dai.Pipeline() as pipeline:
    camB = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_B)
    camC = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_C)
    outB = camB.requestOutput((640, 400))
    outC = camC.requestOutput((640, 400))

    stitching = pipeline.create(dai.beta.node.Stitching).build([outB, outC])
```

#### C++

```cpp
dai::Pipeline pipeline;
auto camB = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_B);
auto camC = pipeline.create<dai::node::Camera>()->build(dai::CameraBoardSocket::CAM_C);
auto* outB = camB->requestOutput(std::make_pair(640, 400));
auto* outC = camC->requestOutput(std::make_pair(640, 400));

auto stitching = pipeline.create<dai::beta::node::Stitching>()->build({outB, outC});
```

The mode examples below reuse `pipeline`, `camB` and `camC` from this setup. Choose one example to replace the output requests and
Stitching node creation above. In Python, keep that code inside the `with dai.Pipeline()` block. The complete examples later on
this page also show how to start the pipeline and display the output.

`stitching.build(numInputs)` also accepts a number instead of a list of outputs. The inputs are then left unlinked and you can
feed them from host queues through `stitching.inputs["input0"]`, `stitching.inputs["input1"]`, and so on.

### Inputs and outputs

 * `inputs` - a map with one input per stitched source, named `input0` to `inputN-1` in the order of the outputs passed to
   `build()`. The inputs are time-aligned by an internal
   [Sync](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/sync.md) subnode, see
   [Synchronization](#Synchronization).
 * `out` - the stitched image, an `ImgFrame` of type `BGR888i`. Its
   [ImgTransformation](https://docs.luxonis.com/software-v3/depthai/depthai-components/messages/img_transformations.md) describes
   the virtual camera that rendered the image, so the panorama or the bird's-eye view can be placed in space like any other camera
   frame, see [Output camera model](#Output%20camera%20model).

## Panorama

`dai.beta.node.Stitching.Mode.PANORAMA` (the default) warps every input onto a common projection surface and combines the images
into one panorama. Choose the projection surface with `stitching.setCameraModel()`, then choose how to position the inputs: [from
calibration](#Panorama%20from%20calibration) or [from image content](#Panorama%20from%20image%20content).

### Camera model

`stitching.setCameraModel()` selects the surface the panorama is rendered on. It takes a
[`dai.CameraModel`](https://docs.luxonis.com/software-v3/depthai/depthai-components/messages/img_transformations.md) value, but
only the three values that name a projection surface are accepted:

| `dai.CameraModel` value | Surface | When to use it |
| --- | --- | --- |
| `dai.CameraModel.Equirectangular` (default) | sphere | Full horizontal and vertical coverage, the same default as OpenCV.
Straight lines bend near the top and bottom. |
| `dai.CameraModel.Cylindrical` | cylinder | Cameras rotated around one axis (a ring of cameras, a 360° rig). Vertical lines stay
straight, the panorama can span the full 360°. |
| `dai.CameraModel.Perspective` | pinhole plane | Two or three cameras with a narrow combined field of view. Straight lines stay
straight, but the canvas grows quickly with the field of view and cannot exceed 180°. |

`dai.CameraModel.Fisheye` and `dai.CameraModel.RadialDivision` are rejected with an error: they describe lens distortions of the
pinhole plane, not a surface to render on.

The first three diagrams are the panorama surfaces selected by `stitching.setCameraModel()`. The fourth shows the separate
`dai.beta.node.Stitching.Mode.PLANAR_PROJECTION` mode, which projects the inputs onto a physical plane in the scene and then
renders that plane through a virtual camera.

Five OAK4 devices in a ring, composed from their calibration onto a cylinder (`dai.CameraModel.Cylindrical`). The panorama spans
the full 360° and vertical lines stay straight.

### Panorama from calibration

With `stitching.setUseInputCalibration(True)`, the node reads the intrinsics and rotation from each input's
`dai.ImgTransformation`. The first synchronized group produces a panorama without feature matching, and the composition is reused
for following groups.

Cameras do not need to overlap during stitching, but they must be calibrated with respect to each other. Inputs must be
undistorted, as shown by `enableUndistortion=True` in the output requests below.

Camera translation is ignored: all camera centers are treated as coincident. This approximation works well when the scene is far
away compared with the distance between cameras; nearby objects can remain misaligned.

#### Python

```python
outB = camB.requestOutput((640, 400), enableUndistortion=True)
outC = camC.requestOutput((640, 400), enableUndistortion=True)
stitching = pipeline.create(dai.beta.node.Stitching).build([outB, outC])

stitching.setMode(dai.beta.node.Stitching.Mode.PANORAMA)
stitching.setUseInputCalibration(True)
stitching.setCameraModel(dai.CameraModel.Cylindrical)        # Equirectangular (default), Cylindrical or Perspective
stitching.setSeamFinder(dai.beta.node.Stitching.SeamFinder.GRAPHCUT_COLOR)
stitching.setMaxPanoramaSize(4096, 2048)                      # reject canvases larger than this
```

#### C++

```cpp
auto* outB = camB->requestOutput(std::make_pair(640, 400), std::nullopt, dai::ImgResizeMode::CROP, std::nullopt, true);
auto* outC = camC->requestOutput(std::make_pair(640, 400), std::nullopt, dai::ImgResizeMode::CROP, std::nullopt, true);
auto stitching = pipeline.create<dai::beta::node::Stitching>()->build({outB, outC});

stitching->setMode(dai::beta::node::Stitching::Mode::PANORAMA);
stitching->setUseInputCalibration(true);
stitching->setCameraModel(dai::CameraModel::Cylindrical);     // Equirectangular (default), Cylindrical or Perspective
stitching->setSeamFinder(dai::beta::node::Stitching::SeamFinder::GRAPHCUT_COLOR);
stitching->setMaxPanoramaSize(4096, 2048);                    // reject canvases larger than this
```

### Panorama from image content

With `stitching.setUseInputCalibration(False)` (the default), the node wraps OpenCV's `cv::Stitcher`, detects features in the
overlaps and estimates the camera rotations from them. No calibration is needed, but neighbouring cameras must overlap and the
scene must have enough texture.

#### Python

```python
outB = camB.requestOutput((640, 400))
outC = camC.requestOutput((640, 400))
stitching = pipeline.create(dai.beta.node.Stitching).build([outB, outC])

stitching.setMode(dai.beta.node.Stitching.Mode.PANORAMA)
stitching.setUseInputCalibration(False)
stitching.setCameraModel(dai.CameraModel.Cylindrical)        # Equirectangular (default), Cylindrical or Perspective
stitching.setSeamFinder(dai.beta.node.Stitching.SeamFinder.GRAPHCUT_COLOR)
stitching.setContinuous(False)                               # estimate once, then reuse the best transform
stitching.setEstimationFrames(10)
stitching.setPanoConfidenceThreshold(1.0)
stitching.setMaxPanoramaSize(4096, 2048)                     # reject degenerate registrations
```

#### C++

```cpp
auto* outB = camB->requestOutput(std::make_pair(640, 400));
auto* outC = camC->requestOutput(std::make_pair(640, 400));
auto stitching = pipeline.create<dai::beta::node::Stitching>()->build({outB, outC});

stitching->setMode(dai::beta::node::Stitching::Mode::PANORAMA);
stitching->setUseInputCalibration(false);
stitching->setCameraModel(dai::CameraModel::Cylindrical);     // Equirectangular (default), Cylindrical or Perspective
stitching->setSeamFinder(dai::beta::node::Stitching::SeamFinder::GRAPHCUT_COLOR);
stitching->setContinuous(false);                              // estimate once, then reuse the best transform
stitching->setEstimationFrames(10);
stitching->setPanoConfidenceThreshold(1.0);
stitching->setMaxPanoramaSize(4096, 2048);                    // reject degenerate registrations
```

#### Visual registration

These settings control how often feature registration runs and which candidates it accepts:

 * `stitching.setContinuous(False)` (default): the node evaluates registration candidates from synchronized input groups, keeps
   the registration with the strongest geometrically consistent feature-match score, and then uses that fixed transform for all
   following frames. No panorama is emitted while the candidates are evaluated. Use this when the cameras are rigidly mounted.
 * `stitching.setContinuous(True)`: the registration is repeated for every synchronized group. Slow, but tolerates cameras that
   move relative to each other.
 * `stitching.setEstimationFrames(10)`: sets how many complete, valid registration candidates to evaluate before fixing the
   transform (default 10). Only applies when continuous registration is disabled. Failed registrations, oversized candidates and
   candidates that omit an input do not count.
 * `stitching.setPanoConfidenceThreshold(1.0)`: sets the confidence threshold used by OpenCV when selecting matching inputs
   (default 1.0). Continuous registration can produce a panorama from the confidently matched subset. With continuous registration
   disabled, a candidate that omits any input is rejected instead.

Registration can fail when overlap or texture is insufficient. A wide camera baseline and nearby objects can also cause problems.
Calibrated composition avoids feature matching, but its coincident-camera-center assumption still limits alignment of nearby
objects.

The shared [panorama size limit](#Panorama%20size%20limit) also applies to these registrations. To repeat a fixed registration
after moving the cameras, see [Resetting the composition](#Resetting%20the%20composition).

### Panorama size limit

`stitching.setMaxPanoramaSize(width, height)` limits the output canvas in pixels for both calibrated and image-content panoramas.
A canvas that exceeds either dimension is rejected before composition allocates it; the node does not resize it to fit. The size
is unbounded by default. Setting a limit is especially useful for image-content registration, where a degenerate feature match can
otherwise produce a huge canvas.

This setting does not apply to planar projection. Use `stitching.setMaxViewSize(width, height)` to bound its automatically
computed view.

## Planar projection

`dai.beta.node.Stitching.Mode.PLANAR_PROJECTION` projects every input onto a plane given in the common origin frame of the inputs
and renders the plane from a virtual pinhole camera that looks straight at it. The usual plane is the ground, which gives a
bird's-eye (top-down) view of the surroundings of a robot or a vehicle. The projection is driven purely by the calibration carried
in the frames, so the cameras do not need to overlap, and the inputs are projected through their full camera model, so they do not
need to be undistorted.

Inputs: `CAM_B` and `CAM_C` of three OAK4 devices placed around a room, tilted towards the floor.

Result: A bird's-eye view of the floor, with the calibration chart in the middle. Shelves and chairs appear stretched because they
rise above the projection plane.

### Configuration

Using the cameras from [How to place it](#How%20to%20place%20it), request their outputs and define the plane to render:

#### Python

```python
outB = camB.requestOutput((640, 400))
outC = camC.requestOutput((640, 400))
stitching = pipeline.create(dai.beta.node.Stitching).build([outB, outC])

stitching.setMode(dai.beta.node.Stitching.Mode.PLANAR_PROJECTION)
stitching.setPlane(
    dai.Point3f(0, 1.0, 0),  # a point on the plane: the floor 1 m below the origin camera (Y points down)
    dai.Point3f(0, 1, 0),    # plane normal, any length
    dai.LengthUnit.METER,
)
stitching.setMaxRange(4.0, dai.LengthUnit.METER)  # ignore the plane further than this from a camera
stitching.setMinIncidenceAngle(5.0)               # drop rays that graze the plane at less than 5 degrees
stitching.setMaxViewSize(1280, 1280)              # upper bound of the automatically computed view
```

#### C++

```cpp
auto* outB = camB->requestOutput(std::make_pair(640, 400));
auto* outC = camC->requestOutput(std::make_pair(640, 400));
auto stitching = pipeline.create<dai::beta::node::Stitching>()->build({outB, outC});

stitching->setMode(dai::beta::node::Stitching::Mode::PLANAR_PROJECTION);
stitching->setPlane(
    dai::Point3f(0, 1.0f, 0),  // a point on the plane: the floor 1 m below the origin camera (Y points down)
    dai::Point3f(0, 1, 0),     // plane normal, any length
    dai::LengthUnit::METER);
stitching->setMaxRange(4.0f, dai::LengthUnit::METER);  // ignore the plane further than this from a camera
stitching->setMinIncidenceAngle(5.0f);                 // drop rays that graze the plane at less than 5 degrees
stitching->setMaxViewSize(1280, 1280);                 // upper bound of the automatically computed view
```

#### Projection plane

`stitching.setPlane(point, normal, unit)` is required. The plane is expressed in the coordinate system the input extrinsics refer
to: the device's origin camera for a single device, or the common origin for calibrated multiple devices. The axes are X right, Y
down and Z forward; see [Coordinate
Systems](https://docs.luxonis.com/software-v3/depthai/depthai-components/coordinate-systems.md). All cameras must look at the
plane from the same side; it does not have to be horizontal.

#### Virtual camera and output size

By default, the node intersects each input's field of view with the plane and places a virtual camera so that all footprints fit.
The output resolution is derived from the inputs and bounded by `stitching.setMaxViewSize(width, height)` (default 1920x1920).

To render a fixed region at a fixed resolution, pass a `dai.beta.node.Stitching.VirtualCamera` to `stitching.setView(view)`. It
defines the pose relative to the origin frame, intrinsics, width and height. Use `stitching.setViewAuto()` to return to automatic
view selection.

#### Range and grazing angles

`stitching.setMaxRange()` (default 10 m) and `stitching.setMinIncidenceAngle()` (default 5°) cut off the part of the plane near
the horizon, where a few image pixels would be stretched over a large area.

Overlapping inputs use the [seam finder](#Seam%20finder) selected by `stitching.setSeamFinder()`.

## Seam finder

`stitching.setSeamFinder()` controls how the node handles overlapping inputs. The node either copies them in input order or
estimates a seam through the overlap. Any seam finder other than `dai.beta.node.Stitching.SeamFinder.NONE` also enables exposure
compensation and multiband blending, which hides brightness differences between the cameras. Seam estimation costs processing
time: in our measurements a five-camera 640x400 cylindrical panorama ran at the camera frame rate without seams and at roughly 60
% of that with seams.

 * Top · No seam finding: Red circles highlight abrupt transitions between camera images.
 * Bottom · Graph cut: Green circles show the corresponding regions with seam finding, exposure compensation and blending.

### No seam

`dai.beta.node.Stitching.SeamFinder.NONE` copies the inputs directly, without exposure compensation or blending. Later inputs
replace earlier ones in overlapping regions. This is the fastest option.

### Voronoi

`dai.beta.node.Stitching.SeamFinder.VORONOI` finds a seam from the Voronoi diagram of the overlapping images: each pixel goes to
the input whose border is further away. It is inexpensive and works well when the calibration is accurate.

### Dynamic programming

`dai.beta.node.Stitching.SeamFinder.DP_COLOR` finds a seam along the path where the two inputs differ least in color.
`dai.beta.node.Stitching.SeamFinder.DP_COLOR_GRAD` also considers the image gradients.

### Graph cut

`dai.beta.node.Stitching.SeamFinder.GRAPHCUT_COLOR` (the default) minimizes the color difference across the whole overlap.
`dai.beta.node.Stitching.SeamFinder.GRAPHCUT_COLOR_GRAD` also considers the image gradients. These options provide the best
quality but are the most expensive.

## Running on the device

By default the node runs on the host, where it needs DepthAI built with OpenCV support. On RVC4 devices `setRunOnHost(False)`
moves the stitching and its Sync subnode onto the device, so the cameras never leave the device as separate streams:

```python
stitching = pipeline.create(dai.beta.node.Stitching).build(outputs)
stitching.setRunOnHost(False)
```

On RVC2 the node cannot run on the device and falls back to the host automatically. Inputs from several devices can only be
stitched on the host.

## Synchronization

The inputs are grouped by an internal [Sync](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/sync.md)
subnode, exposed as `stitching.sync`, that runs on the same side (host or device) as the stitching itself. `setSyncThreshold()`
sets the maximal timestamp spread of a group (the Sync default is 10 ms); everything else, such as the timestamp source or the
number of sync attempts, is configured on the subnode directly.

```python
from datetime import timedelta

stitching.setSyncThreshold(timedelta(milliseconds=50))
stitching.sync.setSyncAttempts(-1)  # never give up on a group
```

Cameras of one device share one clock, so a threshold of one frame period works for them. In multi-device setups the devices need
a shared clock first, see [Multi-device setups](#Multi-device%20setups).

## Single device with several camera modules

The straightforward use of the node is one FFC baseboard with several camera modules (CCMs) mounted in the directions you need:
side by side for a wide field of view, around a mast for a 360° ring, or on the corners of a vehicle for a surround view. All
modules share one clock, so their frames arrive synchronized, and the stitching can run on the device itself.

### Calibration

Calibrated panorama composition and planar projection both rely on the rotation between the camera modules stored in the device
calibration. A freshly assembled FFC rig does not have one, so calibrate it before you stitch:

 * Mount the modules rigidly and make sure neighbouring modules overlap by a part of their field of view. The calibration is found
   from the common image content, so modules that do not overlap with any other module cannot be calibrated this way.
 * Open the device in the [OAK Viewer](https://docs.luxonis.com/software-v3/depthai/tools/oak-viewer.md) (version 4.2.0 or newer),
   enter the approximate positions and rotations of the modules under Camera Extrinsics, and run the FFC Calibration pipeline. It
   refines the extrinsics with [Dynamic
   Calibration](https://docs.luxonis.com/software-v3/depthai/depthai-components/host_nodes/dynamic_calibration.md) from the live
   images and stores the result in the device calibration.

Once calibrated, every camera output carries its extrinsics with respect to the origin camera of the device and the node needs no
further setup.

### Example

A complete calibrated panorama example for two modules on an RVC4 device. With more modules, append their outputs to the list.

#### Python

```python
import depthai as dai
from datetime import timedelta
import cv2

FPS = 30
SOCKETS = [dai.CameraBoardSocket.CAM_B, dai.CameraBoardSocket.CAM_C]

with dai.Pipeline() as pipeline:
    outputs = []
    for socket in SOCKETS:
        camera = pipeline.create(dai.node.Camera).build(socket, sensorFps=FPS)
        # Calibrated composition needs undistorted inputs
        outputs.append(camera.requestOutput((640, 400), fps=FPS, enableUndistortion=True))

    stitching = pipeline.create(dai.beta.node.Stitching).build(outputs)
    stitching.setRunOnHost(False)  # stitch on the RVC4 device
    stitching.setMode(dai.beta.node.Stitching.Mode.PANORAMA)
    stitching.setUseInputCalibration(True)
    stitching.setCameraModel(dai.CameraModel.Perspective)
    stitching.setMaxPanoramaSize(4096, 2048)
    stitching.setSyncThreshold(timedelta(seconds=1 / FPS))
    queue = stitching.out.createOutputQueue()

    pipeline.start()
    while pipeline.isRunning():
        cv2.imshow("panorama", queue.get().getCvFrame())
        if cv2.waitKey(1) == ord("q"):
            break
```

#### C++

```cpp
#include <opencv2/opencv.hpp>
#include "depthai/depthai.hpp"

int main() {
    constexpr float FPS = 30;
    dai::Pipeline pipeline;
    std::vector<dai::Node::Output*> outputs;
    for(auto socket : {dai::CameraBoardSocket::CAM_B, dai::CameraBoardSocket::CAM_C}) {
        auto camera = pipeline.create<dai::node::Camera>()->build(socket, std::nullopt, FPS);
        // Calibrated composition needs undistorted inputs
        outputs.push_back(camera->requestOutput(std::make_pair(640, 400), std::nullopt, dai::ImgResizeMode::CROP, FPS, true));
    }

    auto stitching = pipeline.create<dai::beta::node::Stitching>()->build(outputs);
    stitching->setRunOnHost(false);  // stitch on the RVC4 device
    stitching->setMode(dai::beta::node::Stitching::Mode::PANORAMA);
    stitching->setUseInputCalibration(true);
    stitching->setCameraModel(dai::CameraModel::Perspective);
    stitching->setMaxPanoramaSize(4096, 2048);
    stitching->setSyncThreshold(std::chrono::milliseconds(33));
    auto queue = stitching->out.createOutputQueue();

    pipeline.start();
    while(pipeline.isRunning()) {
        cv::imshow("panorama", queue->get<dai::ImgFrame>()->getCvFrame());
        if(cv::waitKey(1) == 'q') break;
    }
    return 0;
}
```

## Multi-device setups

The node can also stitch cameras that belong to different devices, as long as all of them are part of the same pipeline and the
stitching runs on the host. The devices need a common clock. For calibrated panorama and planar projection, the frames also need
extrinsics expressed in one common coordinate system instead of each device's own origin camera. The following setup uses a
calibrated panorama.

> **Beta API**
> Cross-device calibration (`dai.beta.MultiDeviceCalibrationHandler`, `dai.beta.node.MultiDeviceCalibration`, `Pipeline.setMultiDeviceCalibration()`) is a [Beta feature](https://docs.luxonis.com/software-v3/depthai/examples/beta-features.md). Its API may still change between releases. The Stitching node is also in beta through DepthAI v3.11.0 and is planned to leave beta in v3.12.0.

 1. Shared clock. Synchronize the devices over [PTP](https://docs.luxonis.com/hardware/platform/deploy/data-sync/ptp-sync.md) and
    enable `TIME_PTP` frame sync on every camera, so the frames of all devices carry comparable timestamps and are captured in the
    same slots. Set the Sync subnode to the `SYSTEM` timestamp source, which is the one PTP aligns.
 2. Common coordinate system. Compute the extrinsics between the devices with the beta `MultiDeviceCalibration` node (the cameras
    of the different devices need to see a common scene while it runs) and store the resulting graph in a JSON file through
    `MultiDeviceCalibrationHandler.toJsonFile()`. In the stitching pipeline, load the file and hand the graph to
    `pipeline.setMultiDeviceCalibration()`. Every device then rebases the extrinsics of its frames to the common origin, which is
    what the calibrated panorama composition and the planar projection need.

### Create the calibration file

Run calibration once before the stitching pipeline. This snippet assumes a pipeline with camera outputs from the different
devices, collected in `outputs`:

```python
calibration = pipeline.create(dai.beta.node.MultiDeviceCalibration)
calibration.setSampleCount(10)
for output in outputs:              # outputs of Camera nodes created on the different devices
    calibration.addCamera(output)
control = calibration.inputControl.createInputQueue()
results = calibration.calibrationOutput.createOutputQueue()

pipeline.start()
control.send(dai.beta.MultiDeviceCalibrationControl.start())
result = results.get()
if result.passed:
    result.getHandler().toJsonFile("multi_device_calibration.json")
```

### Stitch the calibrated inputs

The complete example below loads the saved calibration file and stitches five devices on the host.

#### Python

```python
import depthai as dai
from datetime import timedelta
import cv2

FPS = 10
IPS = ["10.0.0.11", "10.0.0.12", "10.0.0.13", "10.0.0.14", "10.0.0.15"]

handler = dai.beta.MultiDeviceCalibrationHandler("multi_device_calibration.json")
pipeline = dai.Pipeline(createImplicitDevice=False)
pipeline.setMultiDeviceCalibration(handler.getGraph())

outputs = []
for ip in IPS:
    device = pipeline.addDevice(dai.DeviceInfo(ip))
    camera = pipeline.create(dai.node.Camera, device).build(dai.CameraBoardSocket.CAM_B, sensorFps=FPS)
    camera.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.TIME_PTP)
    outputs.append(camera.requestOutput((640, 400), dai.ImgFrame.Type.NV12, fps=FPS, enableUndistortion=True))

stitching = pipeline.create(dai.beta.node.Stitching).build(outputs)  # runs on the host
stitching.setMode(dai.beta.node.Stitching.Mode.PANORAMA)
stitching.setUseInputCalibration(True)
stitching.setCameraModel(dai.CameraModel.Cylindrical)
stitching.setMaxPanoramaSize(8192, 4096)
stitching.setSyncThreshold(timedelta(milliseconds=2))
stitching.sync.setTimestampSource(dai.node.Sync.TimestampSource.SYSTEM)  # PTP-aligned timestamps
queue = stitching.out.createOutputQueue(maxSize=4, blocking=False)

pipeline.start()
while pipeline.isRunning():
    cv2.imshow("panorama", queue.get().getCvFrame())
    if cv2.waitKey(1) == ord("q"):
        break
pipeline.stop()
```

#### C++

```cpp
#include <opencv2/opencv.hpp>
#include "depthai/depthai.hpp"

int main() {
    constexpr float FPS = 10;
    const std::vector<std::string> ips = {"10.0.0.11", "10.0.0.12", "10.0.0.13", "10.0.0.14", "10.0.0.15"};

    dai::beta::MultiDeviceCalibrationHandler handler(std::filesystem::path("multi_device_calibration.json"));
    dai::Pipeline pipeline(false);
    pipeline.setMultiDeviceCalibration(handler.getGraph());

    std::vector<dai::Node::Output*> outputs;
    for(const auto& ip : ips) {
        auto device = pipeline.addDevice(dai::DeviceInfo(ip));
        auto camera = pipeline.create<dai::node::Camera>(device)->build(dai::CameraBoardSocket::CAM_B, std::nullopt, FPS);
        camera->initialControl.setFrameSyncMode(dai::CameraControl::FrameSyncMode::TIME_PTP);
        outputs.push_back(camera->requestOutput(std::make_pair(640, 400), dai::ImgFrame::Type::NV12, dai::ImgResizeMode::CROP, FPS, true));
    }

    auto stitching = pipeline.create<dai::beta::node::Stitching>()->build(outputs);  // runs on the host
    stitching->setMode(dai::beta::node::Stitching::Mode::PANORAMA);
    stitching->setUseInputCalibration(true);
    stitching->setCameraModel(dai::CameraModel::Cylindrical);
    stitching->setMaxPanoramaSize(8192, 4096);
    stitching->setSyncThreshold(std::chrono::milliseconds(2));
    stitching->sync->setTimestampSource(dai::node::Sync::TimestampSource::SYSTEM);  // PTP-aligned timestamps
    auto queue = stitching->out.createOutputQueue(4, false);

    pipeline.start();
    while(pipeline.isRunning()) {
        cv::imshow("panorama", queue->get<dai::ImgFrame>()->getCvFrame());
        if(cv::waitKey(1) == 'q') break;
    }
    pipeline.stop();
    return 0;
}
```

## Examples

 * [Stitching panorama](https://github.com/luxonis/depthai-core/blob/main/examples/python/HostNodes/stitching_panorama.py) -
   Calibrated panorama of `CAM_B` and `CAM_C` of one device.
 * [Stitching planar
   projection](https://github.com/luxonis/depthai-core/blob/main/examples/python/HostNodes/stitching_planar_projection.py) -
   Bird's-eye view of `CAM_B` and `CAM_C` projected onto a plane in front of the device.

## Resetting the composition

`stitching.resetTransform()` clears the cached registration and composition state in all stitching modes. The node rebuilds the
state when it processes the next synchronized input group:

 * Image-content panorama: the node runs feature registration again. With continuous registration disabled, it evaluates the
   configured number of valid candidates before emitting another panorama.
 * Calibrated panorama: the node rebuilds its projection maps and, when enabled, seams and exposure compensation from the
   calibration carried by the input frames.
 * Planar projection: the node rebuilds its plane projection maps, seams and exposure compensation using the current configuration
   and input transformations.

Use this after moving the cameras or when the cached composition needs refreshing. For calibrated panorama and planar projection,
the input frames must carry calibration that reflects the new camera poses; resetting does not recalibrate the cameras.

## Output camera model

The stitched `ImgFrame` carries an
[ImgTransformation](https://docs.luxonis.com/software-v3/depthai/depthai-components/messages/img_transformations.md) that
describes the virtual camera which rendered it:

 * `PANORAMA`: `getDistortionModel()` returns the value of `setCameraModel()` (`Equirectangular`, `Cylindrical` or `Perspective`).
   `fx = fy` is the radius of the sphere or cylinder in pixels and `(cx, cy)` is the pixel the panorama Z axis projects to. With
   calibrated composition the extrinsics refer to the destination frame of the inputs (the origin camera of the device, or the
   common origin of a multi-device calibration), centered at the mean of the camera centers. With visual registration the panorama
   is only known relative to its inputs, so the extrinsics are expressed through the first contributing input.
 * `PLANAR_PROJECTION`: `getDistortionModel()` returns `Perspective`, the intrinsics are those of the virtual pinhole camera of
   `setView()`, and the extrinsics give the pose of that camera with respect to the common origin of the inputs.

A `dai.CameraModel` names a projection surface together with a lens distortion model. The panorama is rendered straight onto its
surface, so the output never carries distortion coefficients and `getDistortionModel()` tells the consumer which surface to wrap
the image onto rather than describing a lens. For a cylindrical panorama with `fx = 286` pixels, one pixel column therefore spans
`1 / 286` radians, and a full 360° ring is `2 * pi * 286 = 1797` pixels wide. This metadata lets you place the panorama next to a
[PointCloud](https://docs.luxonis.com/software-v3/depthai/depthai-components/host_nodes/pointcloud.md) of the same cameras or map
directions in the scene to panorama pixels.

```python
frame = queue.get()
transformation = frame.getTransformation()
model = transformation.getDistortionModel()     # dai.CameraModel.Cylindrical
fx = transformation.getIntrinsicMatrix()[0][0]  # radius of the cylinder in pixels
origin = transformation.getExtrinsics()         # toDeviceId / toCameraSocket of the common origin
```

## Reference

### dai::beta::node::Stitching

Kind: class

Stitching node. Combines N time-synced image streams into a single stitched image.

The node runs on the host by default and can run on an RVC4 device when selected with setRunOnHost(false) . Host execution
requires depthai-core OpenCV support. Inputs are fixed at build() time and synced by an internal Sync subnode that follows the
node's execution side, so host-mode sources may come from different devices. Two independent stitching modes are available:
Mode::PANORAMA wraps OpenCV's cv::Stitcher and registers the images from their content, so no calibration is needed, but the
cameras have to overlap.; Mode::PLANAR_PROJECTION projects the images onto a plane given in the common origin frame of the inputs
(bird's-eye view), driven purely by the calibration carried in the messages, so it also works without overlap. All input
transformations must have the same origin camera socket. The stitched frame carries an ImgTransformation describing the virtual
camera that rendered it, so that the image can be placed in space, e.g. wrapped onto a cylinder next to a point cloud. In
Mode::PLANAR_PROJECTION that is the pinhole view of setView() , relative to the common origin of the inputs. In Mode::PANORAMA the
camera model follows setCameraModel() : dai::CameraModel::Perspective, Cylindrical or Equirectangular (see dai::CameraModel for
their projection formulas), the focal length is the radius of the projection surface in pixels and the principal point is the
pixel the panorama Z axis projects to. A panorama composed from the input calibration is expressed in the destination coordinate
system of the inputs, centered at the mean of their camera centers. A visually registered panorama is only known relative to its
inputs, so it is expressed through the first contributing input: in the destination coordinate system of that input's extrinsics
when it carries some, and relative to that input's camera, with an AUTO socket, otherwise.

#### StitchingProperties::Mode Mode

Kind: enum

Stitching mode.

##### PANORAMA

Kind: enum_value

Photo panorama registered from overlapping image content.

##### PLANAR_PROJECTION

Kind: enum_value

Projection of calibrated views onto a plane.

#### StitchingProperties::SeamFinder SeamFinder

Kind: enum

Seam estimation method.

##### NONE

Kind: enum_value

##### VORONOI

Kind: enum_value

##### DP_COLOR

Kind: enum_value

##### DP_COLOR_GRAD

Kind: enum_value

##### GRAPHCUT_COLOR

Kind: enum_value

##### GRAPHCUT_COLOR_GRAD

Kind: enum_value

#### StitchingProperties::Plane Plane

Kind: typedef

A plane the images are projected onto in Mode::PLANAR_PROJECTION , expressed in the common origin frame of the input
transformations.

#### StitchingProperties::VirtualCamera VirtualCamera

Kind: typedef

The pinhole camera Mode::PLANAR_PROJECTION renders the plane from.

#### Subnode < dai::node::Sync > sync

Kind: variable

Internal Sync node time-aligning the inputs.

#### InputMap & inputs

Kind: variable

A map of inputs, one per stitched source. Populated by build() .

#### Output out

Kind: variable

Stitched image, ImgFrame of type BGR888i. Its transformation describes the virtual camera that rendered it, see the class
documentation.

#### Stitching()

Kind: function

#### Stitching(std::unique_ptr< Properties > props)

Kind: function

#### ~Stitching()

Kind: function

#### std::shared_ptr< Stitching > build(const std::vector< Node::Output * > & sources)

Kind: function

Build the node with a fixed set of sources. The number of inputs cannot change afterwards. parameters: sources: Outputs to stitch,
at least two. Inputs are named input0..inputN-1 in this order

#### std::shared_ptr< Stitching > build(size_t numInputs)

Kind: function

Build the node with a fixed number of unlinked inputs, to be fed from host queues. parameters: numInputs: Number of inputs to
stitch, at least two. Inputs are named input0..inputN-1

#### size_t getNumInputs()

Kind: function

Number of inputs the node was built with.

#### void setSyncThreshold(std::chrono::nanoseconds syncThreshold)

Kind: function

Set the maximal interval between messages of a synced group. parameters: syncThreshold: Maximal interval between messages in the
group

#### void setRunOnHost(bool runOnHost)

Kind: function

Specify whether to run on host or an RVC4 device. By default, the node runs on host.

#### bool runOnHost()

Kind: function

Check whether the node is configured to run on host.

#### void setMode(Mode mode)

Kind: function

Set the stitching mode. Mode::PLANAR_PROJECTION additionally needs a plane, see setPlane() .

#### Mode getMode()

Kind: function

#### void setPlane(const Plane & plane)

Kind: function

Plane the images are projected onto, in Mode::PLANAR_PROJECTION . Required in that mode. The plane is expressed in the common
origin frame of the incoming transformations, which is the camera socket their extrinsics are relative to. A plane that all
cameras look at from the same side gives the usual bird's-eye view; the plane does not have to be horizontal.

#### void setPlane(const Point3f & point, const Point3f & normal, LengthUnit unit)

Kind: function

#### std::optional< Plane > getPlane()

Kind: function

#### void setView(const VirtualCamera & view)

Kind: function

Camera the plane is rendered from, in Mode::PLANAR_PROJECTION . By default the view is computed from the content: the node
intersects the field of view of every input with the plane and places a camera looking straight at the plane so that all of the
footprints fit, at a resolution derived from the inputs and bounded by setMaxViewSize() .

#### void setViewAuto()

Kind: function

#### std::optional< VirtualCamera > getView()

Kind: function

#### void setMaxViewSize(uint32_t width, uint32_t height)

Kind: function

Upper bound on the size of the automatically computed view, in pixels. Defaults to 1920x1920.

#### void setMaxRange(float range, LengthUnit unit)

Kind: function

Distance from a camera center beyond which the plane is not painted anymore. Bounds the automatic view and cuts off the region
around the horizon, where a few pixels are stretched over a large part of the plane. Defaults to 10 meters.

#### float getMaxRange(LengthUnit unit)

Kind: function

#### void setMinIncidenceAngle(float degrees)

Kind: function

Smallest angle between a camera ray and the plane for the ray to still be used. Rays hitting the plane at a shallower angle are
heavily stretched, so they are dropped. Defaults to 5 degrees.

#### float getMinIncidenceAngle()

Kind: function

#### void setCameraModel(CameraModel model)

Kind: function

Set the projection surface the images are warped onto. Defaults to Equirectangular (a sphere), same as OpenCV. Only used in
Mode::PANORAMA . parameters: model: dai::CameraModel::Equirectangular, Cylindrical or Perspective parameters:
std::invalid_argument: for any other model

#### CameraModel getCameraModel()

Kind: function

#### void setUseInputCalibration(bool useInputCalibration)

Kind: function

Use the intrinsics and rotations carried by the input ImgTransformations to compose a panorama. All inputs must be expressed
relative to exactly the same destination device ID and camera socket. Translation is ignored, so all camera centers are treated as
coincident. Cylindrical panoramas use the normalized mean input camera Y axis as the cylinder axis. When enabled, panorama
composition starts from the first synchronized group without feature matching or bundle adjustment. With SeamFinder::NONE , warped
inputs are copied in order and later inputs replace earlier ones in overlapping regions. Other seam finders enable seam
estimation, exposure compensation, and multiband blending. The prepared composition is reused for subsequent groups. Only
undistorted inputs are accepted. Only used in Mode::PANORAMA .

#### bool getUseInputCalibration()

Kind: function

Return whether panorama composition uses the calibration carried by the input ImgTransformations.

#### void setContinuous(bool continuous)

Kind: function

Re-estimate the camera parameters on every frame. Only used in Mode::PANORAMA . When true, registration runs for every synced
group, which is slow but tolerates cameras that move relative to each other. When false, registration evaluates
getEstimationFrames() complete candidates without composing them, selects the one with the strongest geometrically consistent
feature-match score, and then starts emitting panoramas using that transform. Projection maps, output regions, seam masks and
exposure parameters are prepared with the first emitted panorama and reused for subsequent groups.

#### bool getContinuous()

Kind: function

#### void setEstimationFrames(uint32_t frames)

Kind: function

Number of complete registration candidates evaluated before the strongest transform is fixed. Failed registrations, oversized
candidates, and candidates that omit an input do not count. No panorama is emitted while the candidates are being evaluated. Only
used when continuous is false.

#### uint32_t getEstimationFrames()

Kind: function

#### void setMaxPanoramaSize(uint32_t width, uint32_t height)

Kind: function

Reject panorama registrations whose projected canvas exceeds this size before OpenCV allocates and composes it. This protects
against degenerate feature matches producing extremely large canvases. By default the size is unbounded.

#### void resetTransform()

Kind: function

Discard the fixed transform and composition state and re-run the estimation. In Mode::PLANAR_PROJECTION the projection maps, seams
and exposure gains are rebuilt from the next synced group. In non-continuous Mode::PANORAMA , registration is repeated and the
fixed maps, regions, seams and exposure parameters are rebuilt.

#### void setPanoConfidenceThreshold(double threshold)

Kind: function

Confidence below which an image is dropped from the panorama.

#### double getPanoConfidenceThreshold()

Kind: function

#### void setSeamFinder(SeamFinder finder)

Kind: function

Set the panorama seam finder. In calibrated panorama mode, NONE selects direct composition; other values also enable exposure
compensation and multiband blending.

#### SeamFinder getSeamFinder()

Kind: function

#### void buildInternal()

Kind: function

#### void run()

Kind: function

### Need assistance?

Head over to [Discussion Forum](https://discuss.luxonis.com/) for technical support or any other questions you might have.