Tutorial Camera Calibration
Camera calibration is the process of estimating intrinsic and/or extrinsic parameters. Intrinsic parameters deal with the camera's internal characteristics, such as, its focal length, skew, distortion, and image center. Extrinsic parameters describe its position and orientation in the world. Knowing intrinsic parameters is an essential first step for 3D computer vision, as it allows you to estimate the scene's structure in Euclidean space and removes lens distortion, which degraces accuracy.
BoofCV provides fully automated calibration from planar targets with square and checkered patterns, that can be easily printed. It is also possible to use 3D calibration targets or other types of calibration grids, provided that the user writes code for detecting the calibration points. This tutorial only discusses the fully automated calibration procedure for planar targets.
Calibration in BoofCV is heavily influenced by Zhengyou Zhang's 1999 paper, "Flexible Camera Calibration By Viewing a Plane From Unknown Orientations". See his webpage below for the paper and more technical and theoretical information on camera calibration. A link is also provided to a popular matlab calibration toolbox.
- Zhang's Camera Calibration
- CalTech's Matlab Calibration Toolbox
- R. Hartley, and A. Zisserman, "Multiple View Geometry in Computer Vision"
- Calibrate Monocular Camera
- Calibrate Stereo Camera
- Remove Lens Distortion
- Rectify Calibrated Stereo
- Rectify Uncalibrated Stereo
In this section, the camera calibration procedure is broken down into steps and explained. Almost identical steps are followed for calibration a single camera or a stereo camera system. First a quick overview:
- Select a pattern, download, and print
- Mount the pattern onto a rigid flat surface
- Take many pictures of the target at different orientations and distances
- Download pictures to compute and select ones that are in focus
- Use provided examples to automatically detect calibration target and compute parameters
- Move calibration file to a safe location
Which calibration target you use is a matter of preference. Currently it is recommended that you use the chessboard pattern since it appears to produce more accurate results. Other types of calibration grids can be used if you provide the location of the calibration points.
Below are two postscript documents of said calibration grids. When using the calibration example code below note how it specifies each pattern's size and the width of each squares. Both black and white squares are counted when specifying the target's grid size. If you are using the provided calibration grids, then no changes are needed. However, if you wish to use your own targets then you will need to tell the software about the new grids characteristics.
- Letter Sized Paper: Chessboard, 5 by 7, 30mm Squares
- Letter Sized Paper: Square Grid, 5 by 7, 30mm Squares
- A4 Sized Paper: Chessboard, 5 by 8, 30mm Squares
- A4 Sized Paper: Square Grid, 5 by 9, 30mm Squares
- A1 Sized Paper: Chessboard, 8 by 12, 60mm Squares
- A1 Sized Paper: Square Grid, 7 by 11 60mm Squares
WHEN PRINTING THE CALIBRATION GRID MAKE SURE YOUR PRINTER DOES NOT RESCALE THE DOCUMENTS!
Failure to do so will cause intrinsic parameters to be off by a scale-factor. Measure with a ruler to make sure each square is 30mm wide
The target needs to be mounted on a flat surface and any warping will decrease calibration accuracy. An ideal surface will be rigid and smooth, for example a table, glass, or marble tile. Cardboard or foam will still work well, but has a tendency to warp over time. Having said that, don't go overboard trying to make a perfectly smooth surface
Before you start taking pictures, make sure the camera has a fixed focal length. The calibration procedure assumes that every picture taken has the same focal length. Many scientific cameras have a fixed focal length so you don't need to worry. Using a tripod to stabilize the camera is also a good idea. Motion blur and being out of focus will throw off calibration.
A quick note on cheap webcams. Some webcams can apparently change focus without changing their focal length. A zoom lens will always change the focal length when at a different zoom. The easiest way to know if autofocus will change your focal length is to put it in manual mode and calibrate at different focus values.
Taking a diverse set of in focus image is essential to calibration. Images should be taken at several different orientations, distances, and locations in the image, while filling up as much of the image as possible. Be sure that the calibration target appears along the image edge and the center. If all the pictures are taken in one region then the results will be biased, even if the residual error is low. The whole target needs to be visible in the image and in some cases the target's border also needs to be visible. Also avoid extreme angles or distances when taking photos. Images used in BoofCV's examples can be found in the BoofCV-Data git repository: https://github.com/lessthanoptimal/BoofCV-Data/tree/master/evaluation/calibration.
See examples directory for sample code. In the evaluation package there is an application which will visualize the results. After calibration is done, look at errors for individual images and see if there are any outliers or images in which the target was not detected. Any targets with unusually large errors should be removed and/or replaced with a better image.
After the example code has run it will save the results into an XML file. Put this XML file is a safe location for future use in your project since. For additional details on the calibration procedure see example wiki page.
Custom Target Detection
In most cases it's easiest and best to use the fully automated algorithms provided in BoofCV which will detect targets automatically. However, there are situations where these automatic algorithms will not work, but you still wish to use the underlying calibration code. It's easy to detect the calibration target's yourself and provide BoofCV with the observed point locations and target description.
Lens distortion can heavily distort features around the image border making them difficult to detect. To overcome this problem the image can be undistorted, making as if an idealized camera is being used. However this can be an expensive operation and some times feature are detected in the distorted image and their position correctly afterwards.
Stereo rectification is the process of distorting two images such that both their epipoles are at infinity, typically along the x-axis. When this happens the epipolar lines are all parallel to each other simplifying the problem of finding feature correspondences to searching along the image axis. Many stereo algorithms require images to be rectified first.
Rectification can be done on calibrated or uncalibrated images. Calibration in this case refers to the stereo baseline (extrinsic parameters between two cameras) to be known. Although in practice it is often required that lens distortion be removed from the images even in the "uncalibrated" case.
The uncalibrated case can be done using automatically detected and associated features, however it is much tricker to get right than the calibrated case. Any small association error will cause a large error in rectification. Even if a state of the art and robust feature is used (e.g. SURF) and matches are pruned using the epipolar constraint, this alone will not be enough. Additional knowledge of the scene needs to be taken in account.