HomeCloudSoftwareHardware
Platform

ON THIS PAGE

  • About Camera Calibration
  • Intrinsic Parameters
  • Extrinsic Parameters
  • Calibration Process
  • Distortion Models
  • Stereo camera recalibration
  • Prerequisites
  • Prepare Charuco Board
  • Display the Charuco Board
  • Calibration
  • Run the Calibration Script
  • Prepare the Board Config
  • Run the Calibration Script
  • Camera Positioning During Calibration
  • Running the Processing Stage
  • Test Depth
  • Troubleshooting
  • ToF recalibration
  • Calibration Procedure

Stereo camera recalibration

Estimate intrinsics and extrinsics of all camera sensors

ToF recalibration

Calibrate for ToF extrinsics

About Camera Calibration

Camera calibration is the process of determining the intrinsic, extrinsic, and distortion parameters of a camera. These parameters enable accurate mapping of 3D points to the 2D sensor and help correct lens distortions.

Intrinsic Parameters

Intrinsic parameters define the internal characteristics of a camera:
  • Focal Length (fx, fy): Sets the field of view. A higher focal length narrows the view, while a lower one widens it.
  • Optical Center (cx, cy): The principal point where the optical axis intersects the sensor. A slight offset (around 20 pixels) is acceptable. This occurs due to minor lens-sensor alignment variations, like small shifts or tilts. Proper calibration compensates for these offsets, so they don't affect the camera's accuracy or performance.
  • Distortion Coefficients: Adjust for lens imperfections:
    • Radial Distortion (k1, k2, k3, k4, k5, k6): Corrects barrel or pincushion effects.
    • Tangential Distortion (p1, p2): Compensates for lens-sensor misalignment.
    • Thin Prism/Skew Distortion (s1, s2, s3, s4, τx, τy): Addresses tilt issues, particularly in wide-angle lenses.

Extrinsic Parameters

Extrinsic parameters describe the camera's position and orientation relative to the scene or other cameras:
  • Rotation: The orientation of the camera.
  • Translation: The camera's position relative to a reference point.
These parameters are essential for applications involving multiple cameras, such as stereo vision, as they allow each camera's image to align with the others.

Calibration Process

Calibration is the process of accurately determining the camera's intrinsic and extrinsic parameters. A robust calibration procedure typically includes:
  1. Marker-Based Calibration: Using Charuco markers (a combination of chessboard and Aruco markers) positioned at various distances and angles for high accuracy.
  2. Controlled Environment: Performing calibration in a stable environment to cover all relevant poses and angles. This approach helps achieve consistently reproducible calibration results.
  3. Model-Specific Customization: Adapting the calibration process to the specific camera model for enhanced reliability.

Distortion Models

Different distortion models are used to correct the unique distortions for each lens type. These models are based on OpenCV's distortion model framework but may be adapted for specific camera configurations.
  • Normal Field of View (NFOV):
    Uses a standard perspective model to correct radial (k1, k2, k3, …) and tangential (p1, p2) distortions.
  • Wide Field of View (WFOV):
    Utilizes an extended perspective model, which can have up to 14 total distortion parameters:
    1. Radial Distortion: k1, k2, k3, k4, k5, k6
    2. Tangential Distortion: p1, p2
    3. Thin Prism: s1, s2, s3, s4
    4. Tilt Parameters: τx, τy
    Note: In practice, we do not activate all 14 parameters. For wide-angle lenses, we enable two additional parameters (often τx and τy) to address the tilt we observe. This extended perspective model handles our wide-field-of-view (WFOV) lenses very well.
  • Fisheye Lenses:
    Employ a specialized model for extreme, circular distortions. We generally avoid fisheye calibration because OpenCV's fisheye support doesn't meet all our needs, and the extended perspective model already covers our WFOV applications effectively.
For additional details, please refer to the OpenCV Documentation on Camera Calibration and Distortion Models.

Stereo camera recalibration

Using the intrinsic, extrinsic, and distortion parameters (as described in the Camera Calibration section), stereo calibration aligns both cameras relative to each other. From these parameters, the system computes rectification matrices, enabling the StereoDepth node to produce accurate stereo disparity and depth estimations.Watching the video below will give you the steps needed to calibrate your own DepthAI. For more information/details on calibration options, see the steps below and also ./calibrate.py --help which will print out all of the calibration options.

Prerequisites

If you don't yet have the depthai repository on your computer, you need to clone it and install the requirements:
Command Line
1git clone https://github.com/luxonis/depthai.git
2cd depthai
3git submodule update --init --recursive
4python3 install_requirements.py

Prepare Charuco Board

We recommend displaying the charuco board on a TV or a large flat monitor. Larger screens are better, as they allow more charuco markers to be visible in the image, typically improving calibration accuracy. Depending on the screen size, we suggest displaying the following charuco board in full-screen:If you have a different screen size, round down to the nearest size.

Display the Charuco Board

When displaying the charuco board, markers and squares should be sharp and clearly visible. Keep in mind:
  • Avoid too bright or dim screens.
  • Avoid direct bright lights or sun on the screen.
  • Display the charuco board in full-screen.
Note the size of the squares on the printed charuco board. This is the SQUARE_SIZE_IN_CM argument for the calibration script.

Calibration

Please select the appropriate calibration procedure for your device:
Compact devices
Modular devices
ArgumentArgument aliasArgument Description
-s--squareSizeCmMeasured square size of the printed charuco board in centimeters
-brd--boardName of the camera (from depthai-boards, not case-sensitive), or path to a custom .json board config
-nx--squaresXNumber of squares in X direction. SquaresX is specified in Prepare Charuco Board, depending on your screen size.
-ny--squaresYNumber of squares in Y direction. SquaresY is specified in Prepare Charuco Board, depending on your screen size.
-cm--cameraModeCamera mode, either perspective (default) or fisheye.
-mdmp--minDetectedMarkersPercentMinimum percentage of detected markers in a frame, to consider the frame valid. Default is 0.5 (50%). If you want to be more strict, you can increase this value, but it can cause longer time to get enough valid frames.
-ep--maxEpipolarErrorMaximum epipolar error in pixels, to consider the frame valid. Default is 0.7. If you want to be more strict, you can decrease this value.
For a full list of arguments and usage examples, run:
Command Line
1python3 calibrate.py --help

Camera Positioning During Calibration

We suggest capturing the calibration from different angles and distances, as it will assist the calibration algorithm in finding the best possible calibration.1. Close to the screen: Ensure the calibration board covers almost the entire field of view (FOV). Take 5 images to cover the entire FOV of the camera:
  • Front view, with the calibration board in the middle of the FOV.
  • Without moving the camera, rotate to align the camera FOV with the calibration board edges. Take 4 images, one for each edge of the screen.
2. Close to the screen, from the side: Capture 4 or more images of the calibration board tilted, covering the majority of the FOV. Move the camera to the top, bottom, left, and right sides of the screen. Varying distances can be used.3. Middle distance: The calibration board should cover about 40% of the FOV. Take 5 images to cover the entire FOV of the camera:
  • Front view, with the calibration board in the middle of the FOV.
  • Similar to the close to the screen position, rotate the camera to align the FOV with the calibration board edges.
4. Far from the screen: The calibration board covers only a small part of the FOV. In total, take 9 images to cover the entire FOV of the camera:
  • Frontal view, with the calibration board in the middle of the FOV.
  • Take 4 images aligning the camera FOV with all 4 edges, similar to the close and middle distance positions.
  • Also, take 4 images aligning with the corners of the screen.

Running the Processing Stage

After capturing images, we can run the processing stage of the calibration. This is done by pressing s key. The script will show epiplolar lines for each image, and you should check if they are aligned correctly. Once all images are checked, the calibration result (if successful) will be flashed on to the device EEPROM. Each captured image is saved in the dataset folder, so you can re-run the calibration process on its own, without having to capture the images again.If you want to re-run the calibration process on the captured images, use the -m process argument:
Command Line
1python3 calibrate.py -s [SQUARE_SIZE_IN_CM] --board [BOARD] -nx [squaresX] -ny [squaresY] -m process
Calibration results are stored inside the resources/ folder and can be used later for testing/debugging purposes. You can also load/flash this local calibration file to the device - see example for more details.

Test Depth

For testing depth quality, use DepthAI Viewer, which can be installed by following the Getting Started instructions.

Troubleshooting

  • If calibration fails with error: High reprojection error!, the usual cause is a misconfigured board config, often due to incorrect specified HFOV of the used camera module.
  • If despite successful calibration, the depth is still incorrect, perhaps your left and right cameras are swapped. Retry the calibration with changed board config, or swap the board sockets the cameras are plugged into.

ToF recalibration

The Time-of-Flight (ToF) calibration is essential for aligning the ToF sensor with other cameras in the system. This procedure focuses on obtaining extrinsic parameters, crucial for ensuring coordinated operation among different cameras.

Calibration Procedure

If you have already installed the DepthAI repository, update it for TOF calibration with the following commands:
Command Line
1git checkout new_tof_calib
2git submodule update --init --recursive
After updating the boards, install the updated DepthAI Python libraries:
Command Line
1python3 .\install_requirements.py
To start the calibration process, run calibrate.py with the appropriate parameters for the Charuco board you are using. For example:
Command Line
1python3 calibrate.py -db -nx 12 -ny 9 -c 1 -cd 0 -s 6 -ms 4.7 -brd OAK-D-SR-POE
Parameters explained:
  • -db: Indicates the default board, meaning you are using Charuco markers.
  • -nx: Number of Charuco markers in the x direction.
  • -ny: Number of Charuco markers in the y direction.
  • -c: Number of pictures taken each time the polygon is displayed (optional).
  • -cd: Countdown time before a picture is taken in seconds (optional).
  • -s: Size of the square around the Charuco marker in centimeters.
  • -ms: Size of the markers in centimeters.
  • -brd: Board of the device (in this case, OAK-D-SR-POE).
If you encounter errors such as division by zero or Failed to detect markers in the image dataset/rgb/rgb_p3_10.png, delete the picture with poor detection of Charuco boards in all camera folders, then run the same code again with the added parameter -m process. This will initiate only the processing stage, so you won't have to retake pictures of the board.
Command Line
1python3 calibrate.py -db -nx 12 -ny 9 -c 1 -cd 0 -s 6 -ms 4.7 -brd OAK-D-SR-POE -m process