Shutter Type
OAK cameras support both rolling shutter and global shutter camera sensors. Here's the broad overview of the differences:- Rolling Shutter - The image is captured by scanning the image from top to bottom. This means that the image is not captured all at once, but one row at a time. This can cause distortion (jello effect) in the image when the camera or the object is moving. Rolling shutter sensors can be higher-resolution without being too expensive.
- Global Shutter - The image is captured all at once. This means that the entire image is captured at the same time, which eliminates the jello effect. Global shutter sensors are typically a lot more expensive at higher resolutions (above 1MP), that's why our current max supported global shutter sensor is 2.3MP (AR0234).
Rolling Shutter
Recommended for applications where higher resolution is needed and the camera or the object isn't moving too fast.In the image below, green marked time periods represent the exposure of each image row, while blue marks the readout of each image row. The exposure and readout of each image row is slightly shifted relative to the previous image row. This leads to distortions caused by the movement of the object during the whole exposure period.
Time difference between starting exposure of each new row ("row time") depends on the sensor, MIPI lanes & config, MIPI speed, and raw bit depth (8/10/12 bits per pixel). Generally, we can estimate it with row_time = 1 / (maxFps * rows). So for IMX378 @ 1080P, it would be row_time = 1/(60 * 1080) = 15.4us.The animation below shows the downside of using rolling shutter while capturing a moving object.
Estimating Jello effect
Jello effect is the distortion in the image when the camera or the object is moving. It's caused by the time difference between the starting exposure of each new row. The jello effect depends on:- Speed of the object
- Object size
- Resolution used
Python
1obj_speed = 0.364 # [fov%/sec]
2obj_height = 0.1 # [%] Object height in the FOV
3width_pix, height_pix = 1920, 1080 # [pix] 1080P resolution
4row_time = 1 / (60 * height_pix) # 15.4 microsec
5
6obj_speed_pix = obj_speed * width_pix # 1920 pix/sec movement
7shift_per_row = obj_speed_pix * row_time # 0.0148[pix/row] shift
8
9# Number of rows, 10% of the FOV => 108 rows * shift
10total_shift = obj_height * height_pix * shift_per_row
11print('Object shift:', total_shift, 'pix') # -> Object shift: 1.16 pix
12print('Relative obj shift', total_shift / width_pix * 100, '%') # -> 0.06 %
13
14width_pix, height_pix = 3840, 2160 # [pix] 4K resolution
15obj_speed_pix = obj_speed * width_pix
16row_time = 1 / (30 * height_pix) # 15.4 microsec, same as 1080P
17shift_per_row = obj_speed_pix * row_time
18total_shift = obj_height * height_pix * shift_per_row
19print('Object shift:', total_shift, 'pix') # -> Object shift: 4.66 pix
20# Relative obj shift
21print(total_shift / width_pix * 100, '%') # -> 0.12%Global Shutter
Recommended for applications where the camera or the object is moving fast, and for stereo camera pair, where jello effect can cause issues with disparity matching. There can still be someIn the image below, green marked time periods represent the exposure of each image row, while blue marks the readout of each image row. The exposure starts and ends at the same time for each image row, while the readout is slightly shifted for every image row. However, shifted readout times of image rows don't affect the quality of the captured image.
Using global shutter all pixels are exposed at the same time, unlike using rolling shutter. It allows a better quality of the captured image, even if the object is moving relatively fast. The animation shows how all rows of the image are captured at once.