# Depth

Depth gives developers metric 3D awareness for mapping, measurement, navigation, and spatial AI—computed fully on-device for low
latency and privacy.

OAK devices support two depth families:

 * [Stereo depth](#Stereo%20Depth) (RVC2, RVC4) — depth from a stereo pair:
    * [Classical Stereo Depth](#Classical%20Stereo%20Depth) —
      [StereoDepth](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/stereo_depth.md) node
    * [GPUStereo](#GPUStereo) (RVC4) —
      [GPUStereo](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/gpu_stereo.md) node
    * [Neural Depth (LENS)](#Neural%20Depth) (RVC4) —
      [NeuralDepth](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/neural_depth.md) node
    * [Neural Assisted Stereo](#Neural%20Assisted%20Stereo) (RVC4) — [Neural Assisted
      Stereo](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/neural_assisted_stereo.md) node
 * [Time of Flight (ToF)](#ToF%20Depth) — active ranging on [ToF-equipped devices](https://docs.luxonis.com/hardware/?f-tof=true).
   API: [ToF](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/tof.md) node; overview continues in the
   section below.

Why do we need depth?

 1. Point clouds: generate colorized point clouds for mapping, inspection, or reconstruction.
 2. Spatial AI: fuse depth with detections/landmarks to get XYZ targets for grasping or avoidance.
 3. Navigation: pair depth with semantics to filter hazards and plan safe motion indoors/outdoors.

## Stereo Depth

| Method | Typical / Ideal Use Case |
| --- | --- |
| Classical Stereo Depth | High-speed robotics, precise distance measurement, outdoor environments, well-textured scenes |
| GPUStereo | High-quality point clouds on RVC4 when depth must run on arbitrary input resolutions |
| Neural Depth (LENS) | Human or hand tracking, challenging environments with low texture or reflections (e.g., garages,
warehouses), highest visual quality depth |
| Neural Assisted Stereo | General-purpose depth, bin picking, object dimensioning, scenes requiring both robustness and high
detail |

### Classical Stereo Depth

[Stereo Depth](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/stereo_depth.md) estimates distance by
comparing two camera views and triangulating pixel differences into real-world measurements. OAK cameras deliver [highly
accurate](https://docs.luxonis.com/hardware/platform/depth/depth-accuracy.md),
[configurable](https://docs.luxonis.com/hardware/platform/depth/configuring-stereo-depth.md),
[RGB-aligned](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/rgbd.md) stereo depth, producing dense,
colorized point clouds.

### RVC2

### RVC4

#### Key Characteristics

 * Industry proven baseline: A hardware-accelerated variation of the well understood and known classical stereo algorithm
   Semi-Global Block Matching (SGBM)
 * Performance: Offers extremely high throughput and low power consumption due to direct hardware implementation.
 * Resolution: Supports up to 1280 pixel horizontal resolution for pixel-level accurate depth maps.
 * Algorithm: Implements a cost aggregation and disparity computation strategy optimized for the underlying silicon architecture

| Strengths | Weaknesses |
| --- | --- |
| Very fast – ~30 ms per frame at full resolution | Requires configuration – different modes needed for different scenes |
| Highly efficient – minimal compute usage, no DSP load, low CPU load | Struggles with low-texture regions where stereo matching
is difficult |
| High detail – detects very small objects and preserves full-resolution depth |  |
| Highly configurable – many tuning parameters for specific use cases |  |

### GPUStereo

On [RVC4](https://docs.luxonis.com/hardware/platform/rvc/rvc4.md),
[GPUStereo](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/gpu_stereo.md) adds a GPU-based stereo option
for synchronized stereo pairs. Its main advantage is that it can run depth on arbitrary input resolutions, which makes it a strong
fit for high-quality point cloud generation.

#### Key Characteristics

 * Concept: GPU-based stereo depth on RVC4 for synchronized stereo pairs.
 * Performance: Generally runs at a similar speed to Classical Stereo Depth while adding support for arbitrary input resolutions.
 * Resolution: Well suited to high-quality point clouds and pipelines where preserving more source-image detail matters.
 * Inputs and outputs: Accepts live camera outputs or already-rectified frames, and produces disparity and depth.

| Strengths | Weaknesses |
| --- | --- |
| Arbitrary input resolutions – useful when point cloud quality depends on source detail | RVC4 only – unavailable on earlier
platforms |
| Similar speed to Classical Stereo – keeps a familiar throughput profile | Smaller configuration surface – fewer tuning controls
than StereoDepth in `3.7.1` |
| Flexible pipeline fit – works well with non-standard or custom stereo input resolutions | Still a stereo matcher – low-texture
or reflective scenes remain challenging |

### Neural Depth

OAK4 devices with [RVC4](https://docs.luxonis.com/hardware/platform/rvc/rvc4.md) builds on the depth accuracy and extends it with
on-device [Neural Depth](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/neural_depth.md), which can run on
its own or complement stereo depth to improve performance in challenging scenes.

#### Key Characteristics

 * Concept: The LENS (Luxonis Edge Neural Stereo) model, a proprietary Luxonis architecture, utilizing state-of-the-art techniques
   in neural stereo matching. It runs fully on device utilizing some of the on board AI compute
 * Mechanism: A deep neural network (DNN) * inputs: stereo pair rectified left and rectified right
    * outputs: disparity map, confidence map, edge detection map
 * Resolution: offers 5 variants ranging from very small resolution (fast) to full resolution (slow) to manage the computational
   cost inherent in DNN inference, this allows users to choose the best tradeoff for their use case.

| Strengths | Weaknesses |
| --- | --- |
| Best visual quality – produces the most visually appealing depth maps | Higher latency / lower FPS – for Larger models |
| Excellent object segmentation – clearly separates objects from the background | Small object detail loss – small objects may
blend with the background when using smaller model variants |
| High fill rate – produces dense depth maps with minimal missing areas | Overfilling – distant regions (e.g., sky) may be
incorrectly filled with depth |
| Lowest overall depth error among the available methods | High compute usage – larger models consume significant AI processing
resources |
| Works with passive stereo setups |  |

### Neural Assisted Stereo

For the best of both worlds, combination of Classical and Neural Stereo: [Neural Assisted
Stereo](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/neural_assisted_stereo.md), which pairs stereo
accuracy with learned robustness.

#### RVC4 (Neural Assisted Stereo)

#### Key Characteristics

 * Concept: The depth information from the lower-resolution Neural Stereo output is injected directly into the input images of the
   stereo algorithm which then proceeds with feature matching, cost aggregation and disparity refinement steps of the
   full-resolution data.
 * Performance: On RVC4 `VPP` + `Neural Assisted Stereo` throughput of up to `50 FPS @ 1280x800` can be expected.
 * Produces a high-resolution, dense depth map that benefits from the pixel accuracy of the HW block and the contextual accuracy
   of the neural network.

| Strengths | Weaknesses |
| --- | --- |
| Combines stereo and neural advantages – high-resolution, low-latency stereo enhanced with neural robustness | Potential
hallucinations – combining neural priors with stereo matching can introduce incorrect depth estimates |
| Flexible and tunable – can be adjusted for specific application requirements | Visual fidelity below LENS – depth maps are not
as visually refined as those produced by Neural Depth |
| Better small-object detail than neural stereo |  |
| Higher performance than neural depth – faster and capable of producing higher-resolution depth maps |  |
| Strong edge accuracy – object boundaries are close to Neural Depth quality |  |
| Efficient execution – low DSP and CPU load |  |

## Time of Flight (ToF) Depth

[ToF Depth](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/tof.md) uses a modulated light pulse to measure
the round-trip travel time of photons, giving reliable range on low-texture or low-light surfaces where stereo can struggle. It
can only be utilized on [ToF devices](https://docs.luxonis.com/hardware/?f-tof=true). Unlike stereo, ToF maintains a [consistent
error profile over distance](https://docs.luxonis.com/hardware/platform/depth/depth-accuracy.md), delivering predictable accuracy
across the full range but at a cost of lower FPS.

### RVC2 ToF

### RVC4 ToF

## Performance Metrics

### RVC2 – Stereo

| Stereo depth preset | FPS @800P | FPS @400P | Output resolution | Range (m) |
| --- | --- | --- | --- | --- |
| Default | 16 | 30 | 1/2 Input Resolution | 0–10 |
| Face | 16 | 30 | 1/2 Input Resolution | 0–3 |
| High Detail | 16 | 30 | 1/2 Input Resolution | 0–15 |
| Robotics | 16 | 34 | 1/2 Input Resolution | 0–15 |
| Fast Accuracy | 30 | 90 | Input Resolution | 0–65 |
| Fast Density | 30 | 90 | Input Resolution | 0–65 |
| Fast Density No Subpixel | 50 | 110 | Input Resolution | 0–65 |

### RVC2 – TOF

| Platform | Resolution | Avg FPS | Range (m) |
| --- | --- | --- | --- |
| ToF | 640×480 | 80 | 0.2-5 |

### RVC4 – Stereo

| Preset | Resolution | Avg FPS | Avg Latency | Range (m) |
| --- | --- | --- | --- | --- |
| FAST_ACCURACY | 1280×800 | 55 | 80 ms | 0–65 |
| FAST_DENSITY | 1280×800 | 37 | 150 ms | 0–65 |
| FAST_ACCURACY | 640×400 | 60 | 29 ms | 0–65 |
| FAST_DENSITY | 640×400 | 60 | 39 ms | 0–65 |

### RVC4 – Neural Stereo

| Model Size | Avg FPS | Avg Latency | Range (m) |
| --- | --- | --- | --- |
| 192×120 | 60 | 29 ms | 0–65 |
| 288×180 | 60 | 30 ms | 0–65 |
| 384×240 (nano) | 60 | 38 ms | 0–65 |
| 480×300 (small) | 56 | 45 ms | 0–65 |
| 576×360 (medium) | 38 | 159 ms | 0–65 |
| 768×480 (large) | 22 | 303 ms | 0–65 |
| 864×540 | 18 | 380 ms | 0–65 |
| 960×600 | 14 | 495 ms | 0–65 |
| 1056×660 | 12.5 | 585 ms | 0–65 |
| 1248×780 (extra_large) | 8.5 | 900 ms | 0–65 |

### RVC4 - Neural Assisted Stereo

| Platform | Resolution | Avg FPS | Avg Latency | Range (m) |
| --- | --- | --- | --- | --- |
| NAS default | 1280×800 | 55 | 300 ms | 0–65 |

### RVC4 - GPU Stereo

| Platform | Resolution | Avg FPS | Avg Latency | Range (m) |
| --- | --- | --- | --- | --- |
| GPU Stereo | 1280×800 | 36 | 60 ms | 0–65 |
| GPU Stereo | 640×400 | 60 | 30 ms | 0–65 |

## Depth self-healing

If stereo depth gets distorted (eg. because of serious mechanical impact or calibration drift), DepthAI provides [Dynamic
Calibration](https://docs.luxonis.com/hardware/platform/depth/dynamic-calibration.md) with which you can setup your camera to
continuously "self-heal". Furthermore,
[AutoCalibration](https://docs.luxonis.com/software-v3/depthai/depthai-components/host_nodes/auto_calibration.md) was [introduced
in DepthAI 3.5.0](https://docs.luxonis.com/software-v3/depthai/release-notes.md). Additionally you can also resort to fixing
stereo depth [manually](https://docs.luxonis.com/hardware/platform/depth/manual-calibration.md).

## Code-less quick start with App Store

### OAK Viewer

OAK Viewer is a GUI application that allows you to easily evaluate the camera by visualizing its output in real-time. You can try
it out:

 * [OAK Viewer on Luxonis Hub](https://hub.luxonis.com/control/store/a_BVRxGDVXnoTrkd3NTB2RQa?view=page) or
 * [install it on your host machine](https://docs.luxonis.com/software-v3/depthai/tools/oak-viewer.md).

[Open it on Luxonis Hub](https://hub.luxonis.com/control/store/a_BVRxGDVXnoTrkd3NTB2RQa?view=page)

## Guides and examples

### StereoDepth example

Classical stereo with configurable modes

[StereoDepth example](https://docs.luxonis.com/software-v3/depthai/examples/stereo_depth/stereo_depth.md)

### GPUStereo example

RVC4 GPU stereo for high-quality point clouds at arbitrary input resolutions

[GPUStereo example](https://docs.luxonis.com/software-v3/depthai/examples/stereo_depth/gpu_stereo.md)

### Neural Assisted Stereo

Classical stereo with added NeuralDepth for better performance

[Neural Assisted Stereo](https://docs.luxonis.com/software-v3/depthai/examples/stereo_depth/neural_assisted_stereo.md)

### NeuralDepth examples

Upgrade your point clouds with neural depth

[NeuralDepth examples](https://docs.luxonis.com/software-v3/depthai/examples/neural_depth/neural_depth.md)

### Time of Flight

Extract depth data from our Time of Flight sensor

[Time of Flight](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/tof.md)

### Need assistance?

Head over to [Discussion Forum](https://discuss.luxonis.com/) for technical support or any other questions you might have.