Depth
- Stereo depth (RVC2, RVC4) — depth from a stereo pair:
- Classical Stereo Depth — StereoDepth node
- Neural Depth (LENS) (RVC4) — NeuralDepth node
- Neural Assisted Stereo (RVC4) — Neural Assisted Stereo node
- Time of Flight (ToF) — active ranging on ToF-equipped devices. API: ToF node; overview continues in the section below.
- Point clouds: generate colorized point clouds for mapping, inspection, or reconstruction.
- Spatial AI: fuse depth with detections/landmarks to get XYZ targets for grasping or avoidance.
- Navigation: pair depth with semantics to filter hazards and plan safe motion indoors/outdoors.
Stereo Depth
| Method | Typical / Ideal Use Case |
|---|---|
| Classical Stereo Depth | High-speed robotics, precise distance measurement, outdoor environments, well-textured scenes |
| Neural Depth (LENS) | Human or hand tracking, challenging environments with low texture or reflections (e.g., garages, warehouses), highest visual quality depth |
| Neural Assisted Stereo | General-purpose depth, bin picking, object dimensioning, scenes requiring both robustness and high detail |
Classical Stereo Depth
RVC2
RVC4
Key Characteristics
- Industry proven baseline: A hardware-accelerated variation of the well understood and known classical stereo algorithm Semi-Global Block Matching (SGBM)
- Performance: Offers extremely high throughput and low power consumption due to direct hardware implementation.
- Resolution: Supports up to 1280 pixel horizontal resolution for pixel-level accurate depth maps.
- Algorithm: Implements a cost aggregation and disparity computation strategy optimized for the underlying silicon architecture
| Strengths | Weaknesses |
|---|---|
| Very fast – ~30 ms per frame at full resolution | Requires configuration – different modes needed for different scenes |
| Highly efficient – minimal compute usage, no DSP load, low CPU load | Struggles with low-texture regions where stereo matching is difficult |
| High detail – detects very small objects and preserves full-resolution depth | |
| Highly configurable – many tuning parameters for specific use cases |
Neural Depth
Key Characteristics
- Concept: The LENS (Luxonis Edge Neural Stereo) model, a proprietary Luxonis architecture, utilizing state-of-the-art techniques in neural stereo matching. It runs fully on device utilizing some of the on board AI compute
- Mechanism: A deep neural network (DNN)
- inputs: stereo pair rectified left and rectified right
- outputs: disparity map, confidence map, edge detection map
- Resolution: offers 5 variants ranging from very small resolution (fast) to full resolution (slow) to manage the computational cost inherent in DNN inference, this allows users to choose the best tradeoff for their use case.
| Strengths | Weaknesses |
|---|---|
| Best visual quality – produces the most visually appealing depth maps | Higher latency / lower FPS – for Larger models |
| Excellent object segmentation – clearly separates objects from the background | Small object detail loss – small objects may blend with the background when using smaller model variants |
| High fill rate – produces dense depth maps with minimal missing areas | Overfilling – distant regions (e.g., sky) may be incorrectly filled with depth |
| Lowest overall depth error among the available methods | High compute usage – larger models consume significant AI processing resources |
| Works with passive stereo setups |
Neural Assisted Stereo
RVC4 (Neural Assisted Stereo)
Key Characteristics
- Concept: The depth information from the lower-resolution Neural Stereo output is injected directly into the input images of the stereo algorithm which then proceeds with feature matching, cost aggregation and disparity refinement steps of the full-resolution data.
- Produces a high-resolution, dense depth map that benefits from the pixel accuracy of the HW block and the contextual accuracy of the neural network.
| Strengths | Weaknesses |
|---|---|
| Combines stereo and neural advantages – high-resolution, low-latency stereo enhanced with neural robustness | Potential hallucinations – combining neural priors with stereo matching can introduce incorrect depth estimates |
| Flexible and tunable – can be adjusted for specific application requirements | Visual fidelity below LENS – depth maps are not as visually refined as those produced by Neural Depth |
| Better small-object detail than neural stereo | |
| Higher performance than neural depth – faster and capable of producing higher-resolution depth maps | |
| Strong edge accuracy – object boundaries are close to Neural Depth quality | |
| Efficient execution – low DSP and CPU load |
Time of Flight (ToF) Depth
RVC2 ToF
RVC4 ToF
Depth self-healing
Code-less quick start with App Store
OAK Viewer
OAK Viewer is a GUI application that allows you to easily evaluate the camera by visualizing its output in real-time. You can try it out:
Open it on Luxonis Hub
Guides and examples
Need assistance?
Head over to Discussion Forum for technical support or any other questions you might have.