HomeCloudSoftwareHardware
Platform

ON THIS PAGE

  • Depth Calibration
  • Prerequisites
  • Prepare Charuco Board
  • Display the Charuco Board
  • Calibration
  • Run the Calibration Script
  • Prepare the Board Config
  • Run the Calibration Script
  • Camera Positioning During Calibration
  • Running the Processing Stage
  • Test Depth
  • Troubleshooting
  • ToF Calibration
  • Calibration Procedure

Depth Calibration

Calibrate stereo camera depth perception

ToF Calibration

Calibrate for ToF extrinsics

Depth Calibration

Camera calibration is a process of determining the intrinsic, extrinsic, and distortion parameters of a camera. These are required for a camera to map points in the 3D world to 2D points in an image, and vice versa, to undistorted images, and to determine depth from stereo images.From these parameters, the camera also calculates the rectification matrices used to rectify the images from the stereo camera pair, allowing the StereoDepth node to perform stereo disparity calculation and depth estimation.Watching the video below will give you the steps needed to calibrate your own DepthAI. For more information/details on calibration options, see the steps below and also ./calibrate.py --help which will print out all of the calibration options.

Prerequisites

If you don't yet have the depthai repository on your computer, you need to clone it and install the requirements:
Command Line
1git clone https://github.com/luxonis/depthai.git
2cd depthai
3git submodule update --init --recursive
4python3 install_requirements.py

Prepare Charuco Board

We recommend displaying the charuco board on a TV or a large flat monitor. Larger screens are better, as they allow more charuco markers to be visible in the image, typically improving calibration accuracy. Depending on the screen size, we suggest displaying the following charuco board in full-screen:If you have a different screen size, round down to the nearest size.

Display the Charuco Board

When displaying the charuco board, markers and squares should be sharp and clearly visible. Keep in mind:
  • Avoid too bright or dim screens.
  • Avoid direct bright lights or sun on the screen.
  • Display the charuco board in full-screen.
Note the size of the squares on the printed charuco board. This is the SQUARE_SIZE_IN_CM argument for the calibration script.

Calibration

Please select the appropriate calibration procedure for your device:
Compact devices
Modular devices
ArgumentArgument aliasArgument Description
-s--squareSizeCmMeasured square size of the printed charuco board in centimeters
-brd--boardName of the camera (from depthai-boards, not case-sensitive), or path to a custom .json board config
-nx--squaresXNumber of squares in X direction. SquaresX is specified in Prepare Charuco Board, depending on your screen size.
-ny--squaresYNumber of squares in Y direction. SquaresY is specified in Prepare Charuco Board, depending on your screen size.
-cm--cameraModeCamera mode, either perspective (default) or fisheye.
-mdmp--minDetectedMarkersPercentMinimum percentage of detected markers in a frame, to consider the frame valid. Default is 0.5 (50%). If you want to be more strict, you can increase this value, but it can cause longer time to get enough valid frames.
-ep--maxEpipolarErrorMaximum epipolar error in pixels, to consider the frame valid. Default is 0.7. If you want to be more strict, you can decrease this value.
For a full list of arguments and usage examples, run:
Command Line
1python3 calibrate.py --help

Camera Positioning During Calibration

We suggest capturing the calibration from different angles and distances, as it will assist the calibration algorithm in finding the best possible calibration.1. Close to the screen: Ensure the calibration board covers almost the entire field of view (FOV). Take 5 images to cover the entire FOV of the camera:
  • Front view, with the calibration board in the middle of the FOV.
  • Without moving the camera, rotate to align the camera FOV with the calibration board edges. Take 4 images, one for each edge of the screen.
2. Close to the screen, from the side: Capture 4 or more images of the calibration board tilted, covering the majority of the FOV. Move the camera to the top, bottom, left, and right sides of the screen. Varying distances can be used.3. Middle distance: The calibration board should cover about 40% of the FOV. Take 5 images to cover the entire FOV of the camera:
  • Front view, with the calibration board in the middle of the FOV.
  • Similar to the close to the screen position, rotate the camera to align the FOV with the calibration board edges.
4. Far from the screen: The calibration board covers only a small part of the FOV. In total, take 9 images to cover the entire FOV of the camera:
  • Frontal view, with the calibration board in the middle of the FOV.
  • Take 4 images aligning the camera FOV with all 4 edges, similar to the close and middle distance positions.
  • Also, take 4 images aligning with the corners of the screen.

Running the Processing Stage

After capturing images, we can run the processing stage of the calibration. This is done by pressing s key. The script will show epiplolar lines for each image, and you should check if they are aligned correctly. Once all images are checked, the calibration result (if successful) will be flashed on to the device EEPROM. Each captured image is saved in the dataset folder, so you can re-run the calibration process on its own, without having to capture the images again.If you want to re-run the calibration process on the captured images, use the -m process argument:
Command Line
1python3 calibrate.py -s [SQUARE_SIZE_IN_CM] --board [BOARD] -nx [squaresX] -ny [squaresY] -m process
Calibration results are stored inside the resources/ folder and can be used later for testing/debugging purposes. You can also load/flash this local calibration file to the device - see example for more details.

Test Depth

For testing depth quality, use DepthAI Viewer, which can be installed by following the Getting Started instructions.

Troubleshooting

  • If calibration fails with error: High reprojection error!, the usual cause is a misconfigured board config, often due to incorrect specified HFOV of the used camera module.
  • If despite successful calibration, the depth is still incorrect, perhaps your left and right cameras are swapped. Retry the calibration with changed board config, or swap the board sockets the cameras are plugged into.

ToF Calibration

The Time-of-Flight (ToF) calibration is essential for aligning the ToF sensor with other cameras in the system. This procedure focuses on obtaining extrinsic parameters, crucial for ensuring coordinated operation among different cameras.

Calibration Procedure

If you have already installed the DepthAI repository, update it for TOF calibration with the following commands:
Command Line
1git checkout new_tof_calib
2git submodule update --init --recursive
After updating the boards, install the updated DepthAI Python libraries:
Command Line
1python3 .\install_requirements.py
To start the calibration process, run calibrate.py with the appropriate parameters for the Charuco board you are using. For example:
Command Line
1python3 calibrate.py -db -nx 12 -ny 9 -c 1 -cd 0 -s 6 -ms 4.7 -brd OAK-D-SR-POE
Parameters explained:
  • -db: Indicates the default board, meaning you are using Charuco markers.
  • -nx: Number of Charuco markers in the x direction.
  • -ny: Number of Charuco markers in the y direction.
  • -c: Number of pictures taken each time the polygon is displayed (optional).
  • -cd: Countdown time before a picture is taken in seconds (optional).
  • -s: Size of the square around the Charuco marker in centimeters.
  • -ms: Size of the markers in centimeters.
  • -brd: Board of the device (in this case, OAK-D-SR-POE).
If you encounter errors such as division by zero or Failed to detect markers in the image dataset/rgb/rgb_p3_10.png, delete the picture with poor detection of Charuco boards in all camera folders, then run the same code again with the added parameter -m process. This will initiate only the processing stage, so you won't have to retake pictures of the board.
Command Line
1python3 calibrate.py -db -nx 12 -ny 9 -c 1 -cd 0 -s 6 -ms 4.7 -brd OAK-D-SR-POE -m process