# Depth

Depth gives developers metric 3D awareness for mapping, measurement, navigation, and spatial AI—computed fully on-device for low
latency and privacy.

OAK devices support two depth families:

 * [Stereo depth](#Depth-Stereo%2520Depth) (RVC2, RVC4) — depth from a stereo pair:
   
   * [Classical Stereo Depth](#Depth-Stereo%2520Depth-Classical%2520Stereo%2520Depth) —
     [StereoDepth](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/stereo_depth.md) node
   * [Neural Depth (LENS)](#Depth-Stereo%2520Depth-Neural%2520Depth) (RVC4) —
     [NeuralDepth](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/neural_depth.md) node
   * [Neural Assisted Stereo](#Depth-Stereo%2520Depth-Neural%2520Assisted%2520Stereo) (RVC4) — [Neural Assisted
     Stereo](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/neural_assisted_stereo.md) node

 * [Time of Flight (ToF)](#ToF%20Depth) — active ranging on [ToF-equipped
   devices](https://docs.luxonis.com/hardware.md?f-tof=true). API:
   [ToF](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/tof.md) node; overview continues in the section
   below.

Why do we need depth?

 1. Point clouds: generate colorized point clouds for mapping, inspection, or reconstruction.
 2. Spatial AI: fuse depth with detections/landmarks to get XYZ targets for grasping or avoidance.
 3. Navigation: pair depth with semantics to filter hazards and plan safe motion indoors/outdoors.

## Stereo Depth

| Method | Typical / Ideal Use Case |
| --- | --- |
| **Classical Stereo Depth** | High-speed robotics, precise distance measurement, outdoor environments, well-textured scenes |
| **Neural Depth (LENS)** | Human or hand tracking, challenging environments with low texture or reflections (e.g., garages,
warehouses), highest visual quality depth |
| **Neural Assisted Stereo** | General-purpose depth, bin picking, object dimensioning, scenes requiring both robustness and high
detail |

### Classical Stereo Depth

[Stereo Depth](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/stereo_depth.md) estimates distance by
comparing two camera views and triangulating pixel differences into real-world measurements. OAK cameras deliver [highly
accurate](https://docs.luxonis.com/hardware/platform/depth/depth-accuracy.md),
[configurable](https://docs.luxonis.com/hardware/platform/depth/configuring-stereo-depth.md),
[RGB-aligned](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/rgbd.md) stereo depth, producing dense,
colorized point clouds.

### RVC2

### RVC4

#### Key Characteristics

 * Industry proven baseline: A hardware-accelerated variation of the well understood and known classical stereo algorithm
   Semi-Global Block Matching (SGBM)
 * Performance: Offers extremely high throughput and low power consumption due to direct hardware implementation.
 * Resolution: Supports up to 1280 pixel horizontal resolution for pixel-level accurate depth maps.
 * Algorithm: Implements a cost aggregation and disparity computation strategy optimized for the underlying silicon architecture

| Strengths | Weaknesses |
| --- | --- |
| **Very fast** – ~30 ms per frame at full resolution | **Requires configuration** – different modes needed for different scenes |
| **Highly efficient** – minimal compute usage, no DSP load, low CPU load | **Struggles with low-texture regions** where stereo
matching is difficult |
| **High detail** – detects very small objects and preserves full-resolution depth | |
| **Highly configurable** – many tuning parameters for specific use cases | |

### Neural Depth

OAK4 devices with [RVC4](https://docs.luxonis.com/hardware/platform/rvc/rvc4.md) builds on the depth accuracy and extends it with
on-device [Neural Depth](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/neural_depth.md), which can run on
its own or complement stereo depth to improve performance in challenging scenes.

#### Key Characteristics

 * Concept: The LENS (Luxonis Edge Neural Stereo) model, a proprietary Luxonis architecture, utilizing state-of-the-art techniques
   in neural stereo matching. It runs fully on device utilizing some of the on board AI compute
 * Mechanism: A deep neural network (DNN)
   * inputs: stereo pair rectified left and rectified right
   * outputs: disparity map, confidence map, edge detection map
 * Resolution: offers 5 variants ranging from very small resolution (fast) to full resolution (slow) to manage the computational
   cost inherent in DNN inference, this allows users to choose the best tradeoff for their use case.

| Strengths | Weaknesses |
| --- | --- |
| **Best visual quality** – produces the most visually appealing depth maps | **Higher latency / lower FPS** – for Larger models |
| **Excellent object segmentation** – clearly separates objects from the background | **Small object detail loss** – small objects
may blend with the background when using smaller model variants |
| **High fill rate** – produces dense depth maps with minimal missing areas | **Overfilling** – distant regions (e.g., sky) may be
incorrectly filled with depth |
| **Lowest overall depth error** among the available methods | **High compute usage** – larger models consume significant AI
processing resources |
| **Works with passive stereo setups** | |

### Neural Assisted Stereo

For the best of both worlds, combination of Classical and Neural Stereo: [Neural Assisted
Stereo](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/neural_assisted_stereo.md), which pairs stereo
accuracy with learned robustness.

#### RVC4 (Neural Assisted Stereo)

#### Key Characteristics

 * Concept: The depth information from the lower-resolution Neural Stereo output is injected directly into the input images of the
   stereo algorithm which then proceeds with feature matching, cost aggregation and disparity refinement steps of the
   full-resolution data.
 * Produces a high-resolution, dense depth map that benefits from the pixel accuracy of the HW block and the contextual accuracy
   of the neural network.

| Strengths | Weaknesses |
| --- | --- |
| **Combines stereo and neural advantages** – high-resolution, low-latency stereo enhanced with neural robustness | **Potential
hallucinations** – combining neural priors with stereo matching can introduce incorrect depth estimates |
| **Flexible and tunable** – can be adjusted for specific application requirements | **Visual fidelity below LENS** – depth maps
are not as visually refined as those produced by Neural Depth |
| **Better small-object detail than neural stereo** | |
| **Higher performance than neural depth** – faster and capable of producing higher-resolution depth maps | |
| **Strong edge accuracy** – object boundaries are close to Neural Depth quality | |
| **Efficient execution** – low DSP and CPU load | |

## Time of Flight (ToF) Depth

[ToF Depth](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/tof.md) uses a modulated light pulse to measure
the round-trip travel time of photons, giving reliable range on low-texture or low-light surfaces where stereo can struggle. It
can only be utilized on [ToF devices](https://docs.luxonis.com/hardware.md?f-tof=true). Unlike stereo, ToF maintains a [consistent
error profile over distance](https://docs.luxonis.com/hardware/platform/depth/depth-accuracy.md), delivering predictable accuracy
across the full range but at a cost of lower FPS.

### RVC2 ToF

### RVC4 ToF

## Depth self-healing

If stereo depth gets distorted (eg. because of serious mechanical impact or calibration drift), DepthAI provides [Dynamic
Calibration](https://docs.luxonis.com/hardware/platform/depth/dynamic-calibration.md) with which you can setup your camera to
continuously "self-heal". Furthermore,
[AutoCalibration](https://docs.luxonis.com/software-v3/depthai/depthai-components/host_nodes/auto_calibration.md) was [introduced
in DepthAI 3.5.0](https://docs.luxonis.com/software-v3/depthai/release-notes.md). Additionally you can also resort to fixing
stereo depth [manually](https://docs.luxonis.com/hardware/platform/depth/manual-calibration.md).

## Code-less quick start with App Store

### OAK Viewer

OAK Viewer is a GUI application that allows you to easily evaluate the camera by visualizing its output in real-time. You can try
it out:

 * [OAK Viewer on Luxonis Hub](https://hub.luxonis.com/control/store/a_BVRxGDVXnoTrkd3NTB2RQa?view=page) or
 * [install it on your host machine](https://docs.luxonis.com/software-v3/depthai/tools/oak-viewer.md).

[Open it on Luxonis Hub](https://hub.luxonis.com/control/store/a_BVRxGDVXnoTrkd3NTB2RQa?view=page)

## Guides and examples

### StereoDepth example

Classical stereo with configurable modes

[StereoDepth example](https://docs.luxonis.com/software-v3/depthai/examples/stereo_depth/stereo_depth.md)

### Neural Assisted Stereo

Classical stereo with added NeuralDepth for better performance

[Neural Assisted Stereo](https://docs.luxonis.com/software-v3/depthai/examples/stereo_depth/neural_assisted_stereo.md)

### NeuralDepth examples

Upgrade your point clouds with neural depth

[NeuralDepth examples](https://docs.luxonis.com/software-v3/depthai/examples/neural_depth/neural_depth.md)

### Time of Flight

Extract depth data from our Time of Flight sensor

[Time of Flight](https://docs.luxonis.com/software-v3/depthai/depthai-components/nodes/tof.md)

### Need assistance?

Head over to [Discussion Forum](https://discuss.luxonis.com/) for technical support or any other questions you might have.