Difference between revisions of "Tutorial Camera Calibration"

From BoofCV
m
Line 1: Line 1:
 
= Camera Calibration =
 
= Camera Calibration =
  
Camera calibration is the process of estimating the camera intrinsic and/or extrinsics parameters.  Intrinsic parameters deal with camera specific features such as its focal length, skew, distortion, and image center. Extrinsic parameters describe its position and orientation in the world.  For the remainder of this page we will focus on estimating intrinsic parameters.  Calibration is an essential first step for most 3D computer vision and removing lens distortion can often improve some times improve object detection accuracy.
+
Camera calibration is the process of estimating the camera intrinsic and/or extrinsic parameters.  Intrinsic parameters deal with camera specific features such as its focal length, skew, distortion, and image center. Extrinsic parameters describe its position and orientation in the world.  For the remainder of this page we will focus on estimating intrinsic parameters.  Calibration is an essential first step for 3D computer vision and removing lens distortion can help improve object detection accuracy.
  
BoofCV provides several software tools and support for different calibration target types.  In a typical scenario calibration will be done by viewing a calibration target from different locations and orientations.  Calibration points are then detected in these images.  Since the geometry of the calibration target is known the cameras intrinsic parameters can be estimated to a high level of accuracy.
+
BoofCV provides support for different calibration target types.  Fully automated code is provided for calibrating from planar targets that can be easily printed, but the underlying source code supported more complex 3D calibration targets.  In a typical scenario calibration is done by viewing a calibration target from different locations and orientations.  Calibration points are then detected in these images.  Since the geometry of the calibration target is known the cameras intrinsic parameters can be estimated to a high level of accuracy. For planar targets, two different commonly found patterns composed of squares are provided, chess board and square grid.   
 
 
The most common way to calibrate a camera is using planar calibration targets with squares on them.  BoofCV provides support for two different types of planar calibration targets with different patterns of squares on them, chess board and square grid.  The code itself can be reused to detect similar types of application specific calibration targets.
 
  
 
Example code is provided in the 'example' directory along with data for several different cameras and both target types.  Making it easy to get up and running.  To download the examples either download the full source code to BoofCV or check it out from the repository, but don't forget to also checkout the data directory.
 
Example code is provided in the 'example' directory along with data for several different cameras and both target types.  Making it easy to get up and running.  To download the examples either download the full source code to BoofCV or check it out from the repository, but don't forget to also checkout the data directory.
Line 49: Line 47:
 
# Select a pattern, download, and print
 
# Select a pattern, download, and print
 
# Mount the pattern onto a rigid flat surface
 
# Mount the pattern onto a rigid flat surface
# Take pictures of the target using your camera
+
# Take many pictures of the target at different orientations and distances
 
# Download pictures to compute and select the best ones
 
# Download pictures to compute and select the best ones
 
# Use provided examples to automatically detect calibration target and compute parameters
 
# Use provided examples to automatically detect calibration target and compute parameters
 
# Move outputted calibration file to a safe location
 
# Move outputted calibration file to a safe location
  
Which calibration target you use is a matter of preference.  Currently it is recommended that you use the chessboard pattern since produces more accurate results.  Other types of planar targets can be used by BoofCV if you provide the source code for detecting calibration points on the target.
+
Which calibration target you use is a matter of preference.  Currently it is recommended that you use the chessboard pattern since produces appears to produce more accurate results.  Other types of planar targets not mentioned above can be used by BoofCV if you provide the source code for detecting calibration points on the target.
  
Below are two postscript documents for printing out calibration grid. When using the calibration example code linked to below you need to specify the grid pattern's size and the side of each squares width.  The grid pattern's width and height in squares refers to the number of squares in the outer most box, which is 3 wide and 4 tall in each example.  Even if you swap the rows and columns calibration will work just fine.
+
Below are two postscript documents for printing out calibration grid. When using the calibration example code linked to below you need to specify the grid pattern's size and the side of each squares width.  The grid pattern's width and height in squares refers to the number of squares in the outer most box, which is 3 wide and 4 tall in each example.  Even if you swap the rows and columns, calibration will work just fine.
 
   
 
   
 
* [https://github.com/lessthanoptimal/BoofCV-Data/blob/master/evaluation/calibration/letter_chess.ps| Letter Sized Paper: Chessboard, 3 by 4, 30mm Squares]
 
* [https://github.com/lessthanoptimal/BoofCV-Data/blob/master/evaluation/calibration/letter_chess.ps| Letter Sized Paper: Chessboard, 3 by 4, 30mm Squares]
 
* [https://github.com/lessthanoptimal/BoofCV-Data/blob/master/evaluation/calibration/letter_square.ps| Letter Sized Paper: Square Grid, 3 by 4, 30mm Squares]
 
* [https://github.com/lessthanoptimal/BoofCV-Data/blob/master/evaluation/calibration/letter_square.ps| Letter Sized Paper: Square Grid, 3 by 4, 30mm Squares]
  
'''WHEN PRINTING THE CALIBRATION GRID MAKE SURE YOUR PRINTER DOES NOT RESCALE THE DOCUMENTS!'''  Failure to do so will cause all range estimates (if using stereo) to be off by an unknown scale factor.
+
'''WHEN PRINTING THE CALIBRATION GRID MAKE SURE YOUR PRINTER DOES NOT RESCALE THE DOCUMENTS!'''  Failure to do so will cause calibration errors, which might not be obvious at first.
  
The target needs to be mounted on a surface that is flat as reasonably possible.  Any warping of the paper will decrease its accuracy by introducing localization errors.  A smooth table, glass, or any similar surface works well.  Cardboard or foam is often warped or will warp as time goes on.  Don't go overboard trying to make a perfectly smooth surface  
+
The target needs to be mounted on a surface that is flat as reasonably possible.  In the near by example picture a desk was used.  Any warping of the paper will decrease its accuracy by introducing localization errors.  A smooth table, glass, or any similar surface works well.  Cardboard or foam is often warped or will warp as time goes on.  Don't go overboard trying to make a perfectly smooth surface  
  
 
[[File:ChessboardCalibrationPicture.jpg|thumb|right|Calibration Target]]
 
[[File:ChessboardCalibrationPicture.jpg|thumb|right|Calibration Target]]
Line 69: Line 67:
 
https://github.com/lessthanoptimal/BoofCV-Data/tree/master/evaluation/calibration.
 
https://github.com/lessthanoptimal/BoofCV-Data/tree/master/evaluation/calibration.
  
Example code is provided for calibrating the cameras and applications which can visualize the results, see examples and evaluation directories.  If you use a calibration target that is different from the ones provided be sure to modify the example code with the correct target description.  Look at calibration errors for individual images and see if there are any outliers or images in which the target was not detected.  Any targets with much larger errors should be removed since the image is probably bad.
+
Example code is provided for calibrating the cameras and applications which can visualize the results, see examples and evaluation directories.  If you use a calibration target that is different from the ones provided, be sure to modify the example code with the correct target description.  Look at calibration errors for individual images and see if there are any outliers or images in which the target was not detected.  Any targets with much larger errors should be removed since the image is probably bad.
  
 
After the example code has run it will save the results into an XML file, see code.  Put this XML file is a safe location for future use in your project since.  For additional details on the calibration procedure see example wiki page.
 
After the example code has run it will save the results into an XML file, see code.  Put this XML file is a safe location for future use in your project since.  For additional details on the calibration procedure see example wiki page.

Revision as of 06:09, 2 June 2012

Camera Calibration

Camera calibration is the process of estimating the camera intrinsic and/or extrinsic parameters. Intrinsic parameters deal with camera specific features such as its focal length, skew, distortion, and image center. Extrinsic parameters describe its position and orientation in the world. For the remainder of this page we will focus on estimating intrinsic parameters. Calibration is an essential first step for 3D computer vision and removing lens distortion can help improve object detection accuracy.

BoofCV provides support for different calibration target types. Fully automated code is provided for calibrating from planar targets that can be easily printed, but the underlying source code supported more complex 3D calibration targets. In a typical scenario calibration is done by viewing a calibration target from different locations and orientations. Calibration points are then detected in these images. Since the geometry of the calibration target is known the cameras intrinsic parameters can be estimated to a high level of accuracy. For planar targets, two different commonly found patterns composed of squares are provided, chess board and square grid.

Example code is provided in the 'example' directory along with data for several different cameras and both target types. Making it easy to get up and running. To download the examples either download the full source code to BoofCV or check it out from the repository, but don't forget to also checkout the data directory.

Calibration in BoofCV is heavily influenced by Zhengyou Zhang's 1999 paper, "Flexible Camera Calibration By Viewing a Plane From Unknown Orientations". See his webpage below for the paper and more technical and theoretical information on camera calibration. A link is also provided to a popular matlab calibration toolbox.

References:


Quick Links

Applets

Examples

Calibration Targets:

Calibration Process

In the following section the camera calibration procedure is broken down into steps and explained. First a quick overview:

  1. Select a pattern, download, and print
  2. Mount the pattern onto a rigid flat surface
  3. Take many pictures of the target at different orientations and distances
  4. Download pictures to compute and select the best ones
  5. Use provided examples to automatically detect calibration target and compute parameters
  6. Move outputted calibration file to a safe location

Which calibration target you use is a matter of preference. Currently it is recommended that you use the chessboard pattern since produces appears to produce more accurate results. Other types of planar targets not mentioned above can be used by BoofCV if you provide the source code for detecting calibration points on the target.

Below are two postscript documents for printing out calibration grid. When using the calibration example code linked to below you need to specify the grid pattern's size and the side of each squares width. The grid pattern's width and height in squares refers to the number of squares in the outer most box, which is 3 wide and 4 tall in each example. Even if you swap the rows and columns, calibration will work just fine.

WHEN PRINTING THE CALIBRATION GRID MAKE SURE YOUR PRINTER DOES NOT RESCALE THE DOCUMENTS! Failure to do so will cause calibration errors, which might not be obvious at first.

The target needs to be mounted on a surface that is flat as reasonably possible. In the near by example picture a desk was used. Any warping of the paper will decrease its accuracy by introducing localization errors. A smooth table, glass, or any similar surface works well. Cardboard or foam is often warped or will warp as time goes on. Don't go overboard trying to make a perfectly smooth surface

Calibration Target

Taking a diverse set of sharp image is essential to calibration. Images should be taken at several different orientations, distances, and locations in the image. Be sure that the calibration target appears along the image edge and the center. If all the pictures are taken in one region then the results will be biased, even if the residual error is low. The whole target needs to be visible in the image and in some cases the target's border also needs to be visible. Images used in BoofCV's examples can be found in the BoofCV-Data git repository: https://github.com/lessthanoptimal/BoofCV-Data/tree/master/evaluation/calibration.

Example code is provided for calibrating the cameras and applications which can visualize the results, see examples and evaluation directories. If you use a calibration target that is different from the ones provided, be sure to modify the example code with the correct target description. Look at calibration errors for individual images and see if there are any outliers or images in which the target was not detected. Any targets with much larger errors should be removed since the image is probably bad.

After the example code has run it will save the results into an XML file, see code. Put this XML file is a safe location for future use in your project since. For additional details on the calibration procedure see example wiki page.

Example Code:

  1. Calibrate Monocular Camera
  2. Calibrate Stereo Camera

Custom Target Detection

In most cases it's easiest and best to use the fully automated algorithms provided in BoofCV which will detect targets automatically. However, there are situations where these automatic algorithms will not work, but you still wish to use the underlying calibration code. It's easy to detect the calibration target's yourself and provide BoofCV with the observed point locations and target description.

Example Code:

  1. Calibrate Monocular Camera

Lens Distortion

Lens distortion can heavily distort features around the image border making them difficult to detect. To overcome this problem the image can be undistorted, making as if an idealized camera is being used. However this can be an expensive operation and some times feature are detected in the distorted image and their position correctly afterwards.

Example Code:

  1. Calibrate Given Points

Stereo Rectification

Stereo rectification is the process of distorting two images such that both their epipoles are at infinity, typically along the x-axis. When this happens the epipolar lines are all parallel to each other simplifying the problem of finding feature correspondences to searching along the image axis. Many stereo algorithms require images to be rectified first.

Rectification can be done on calibrated or uncalibrated images. Calibration in this case refers to the stereo baseline (extrinsic parameters between two cameras) to be known. Although in practice it is often required that lens distortion be removed from the images even in the "uncalibrated" case.

The uncalibrated case can be done using automatically detected and associated features, however it is much tricker to get right than the calibrated case. Any small association error will cause a large error in rectification. Even if a state of the art and robust feature is used (e.g. SURF) and matches are pruned using the epipolar constraint, this alone will not be enough. Additional knowledge of the scene needs to be taken in account.

  1. Rectify Calibrated Stereo
  2. Rectify Uncalibrated Stereo