Robotics Vision Core 4 (RVC4)

Robotics Vision Core 4 (RVC4 in short) is the fourth generation of our RVC. Main specs:

Octa-core ARMv8 CPU running Linux
AI: 48 INT8, 12 FP16 TOPS
Computer vision: Stereo depth, frame warp engine, optical flow, feature detection, description matching, template matching
ISP: 5 camera streams, HDR, EIS, 3A
Encoding: 4K @ 240FPS decoding, 4K @ 120FPS encoding for H264 and H265. Decoding also supported for VP9, AV1

To stay up to date, consider subscribing to the RVC4 newsletter.

Computer vision engine

Stereo depth: Max 800P @ 60FPS, 4bit subpixel by default, 64 disparity search. 8-bit confidence map, spatial consistency, occlusion and texture masking, ±3 pixel lines rectification error tolerance
- Maximum stereo resolution is 1280x1280
Image warp engine: Throughput of 1080P @ 240FPS
Optical flow: Semi-dense: 1080P @ 60FPS, Full dense: VGA @ 60 FPS
Feature detection: Harris corner detection, 1080P @ 60FPS
Description matching: ORB calculation and inline matching. Descriptor: 256 bits. Max 1080P, 1ms per 500 descriptors (calculation + matching)
Template matching: Max 1080P, 1.2ms for 500 templates, max 1024 patches per frame

Image Signal Processor

RVC4's Image Signal Processor (ISP) has the following features:

Up to 3 concurrent camera streams (with ISP processing)
- 3x 36MP @ 30FPS or
- 1x 36MP + 1x 72MP @ 30FPS or
- 1x 108MP @ 30FPS
Up to 2 additional raw output streams (no ISP processing)
3A (Auto Exposure, Auto White Balance, Auto Focus)
Supports 18 bpp (bits-per-pixel)
Hardware HDR: Staggered HDR, digital overlap, non-overlap
Image stabilization (EIS), good low-light performance

AI

RVC4 NN model benchmarks (peak performance):

Model name	Size	FPS	Task
YoloV5m	640x640	280	Object detection
YoloV6n	512x288	2340	Object detection
YoloV7-W6	640x640	162	Object detection
ResNet-50	224x224	934	Classification
ViT-Tiny	224x224	650	Classification
BiSeNetv1-MBNV2	512x228	647	Semantic segmentation
eWaSR	512x384	309	Semantic segmentation

AI Power Consumption

RVC4's AI system is designed to be power-efficient and configurable to the user's needs. The AI system can be configured to run at different power levels (FPS speeds), which will affect the performance of the AI system. The following table shows model FPS and [power consumption] at different FPS speeds:

Model name	Low FPS	Medium FPS	High FPS	Max FPS
BiSeNet-MBNV2 (512x288)	80 [0.67 W]	133 [1.04 W]	391 [3.02 W]	647 [5 W]
eWaSR ResNet18 (512x384)	59 [0.79 W]	106 [1.51 W]	229 [3.55 W]	309 [5.25 W]
MobileVit-xxs (224x224)	104 [0.63 W]	199 [1 W]	277 [1.82 W]	488 [3.27 W]
Repvgg_a2 (224x224)	181 [1.07 W]	327 [2.22 W]	466 [3.75 W]	1250 [10.4 W]
ResNet101 (224x224)	115 [1.05 W]	243 [2.4 W]	339 [3.57 W]	718 [8.62 W]
ResNet50-v2-7 (224x224)	145 [0.96 W]	260 [1.9 W]	380 [2.83 W]	934 [7.65 W]
ViT-Tiny patch16 (224x224)	124 [0.7 W]	228 [1.25 W]	300 [1.88 W]	615 [4.27 W]
Yolo6N (512x288)	190 [0.7 W]	307 [1.1 W]	702 [3.8 W]	2340 [7.5 W]
YoloV5M (640x640)	48 [0.93 W]	72 [1.75 W]	212 [6.05 W]	280 [8.4 W]
YoloV7-W6 (640x640)	34 [1.05 W]	60 [2.48 W]	139 [7.45 W]	162 [7.85 W]

Power measurements were taken of the whole RVC4 board during 10 second inference runs. So the AI power consumption is a bit less, as the rest of the chip (mainly CPU) is also consuming power.

Please note that the max FPS values are recorded at peak performance of the RVC4. In most use-cases, the RVC4 are throttled down to avoid overheating.

Jetson comparison

Nvidia's Jetson series is currently the de-facto edge AI platform. We tested the Jetson Orin Nano 8GB (standard, not Super version), which has 40 TOPS (GPU) and 6-core ARM CPU. Below is a 1:1 comparison of RVC4 to Jetson Orin Nano 8GB, both using INT8 precision and the same image shape:

Model name	RVC4 [FPS]	Jetson Orin Nano 8GB [FPS]
InceptionV4, BS1	691	170
InceptionV4, BS32	608	358
ResNet50, BS1	1369	502
ResNet50, BS32	1644	1191
VGG19, BS1	269	183
VGG19, BS32	560	362
Super Resolution, BS1	36*	202
SSD MobileNet V1, BS1	1910	920
SSD MobileNet V1, BS32	2688	2260
UNet Segmentation, BS1	323	142
YoloV3 Tiny, BS1	1342	563

Looking at Jetson Family Benchmarks (at the bottom), Nvidia reports FPS for BS32 models (Batch Size 32, so 32 images getting inferenced all at once). From our own tests, these numbers are realistic, however, their BS1 models (Batch Size 1, so single image) perform ~2x worse than BS32 ones. If you want real-time performance (not +1 sec latency), you will need to use BS1 models.* As the SoC is brand new, the model optimizer is still being updated, and additional layers will be added to be processed on accelerated blocks in the future. For the Super resolution model, a few layers were processed on the CPU. That's why RVC4 performance was low.

Power efficiency

We also measured the power usage of both RVC4 and the Jetson Orin Nano 8GB. We ran both devices at max performance (so MAX FPS for RVC4), and measured the power usage of the whole board. Power usage of Orin Nano fluctuates a lot, so we took the average of the power usage over 10 seconds. We always took a model with Batch Size=1 (so a single image).

Model name	RVC4 [W]	Jetson Orin Nano 8GB [W]
InceptionV4	9.1	12
ResNet50	10.2	13
VGG19	9.5	11
YoloV3 Tiny	9.9	10
YoloV5 M (416x416)	9.3	11

Conclusion: While being 1.9x faster, RVC4 is also 15% more power efficient compared to the Jetson Orin Nano 8GB

Custom applications

Users have full access to the power of the RVC4:

Easy development & deployment of custom containerized apps is possible out-of-the-box via Luxonis Hub
Develop and run fast CV pipelines on top of accelerated hardware blocks using Halide
Interface with GPIOs and communication interfaces
As the RVC4 can also optionally act as a host computer, it is possible to connect other RVC2-based OAK (PoE) cameras to it.

Power consumption

The RVC4 itself has a maximum power peaks of about 25W, which is mainly consumed by the SoC, Qualcomm's QCS8550, that is integrated inside the RVC4.

Max peak: 25W
Max average: 10-15W
Average idle: 1.6W
Booting peak: 12W
Booting average: 2.3W (~15sec)

ON THIS PAGE