Difference between revisions of "Tutorial Camera Calibration"

From BoofCV
Jump to navigationJump to search
m
m (Updating for v0.7)
Line 1: Line 1:
= Camera Calibration =
= Camera Calibration =


Camera calibration is the process of estimating the cameras parameters.  It can either refer to the intrinsic parameters or the extrinsic parameters.  Intrinsic parameters deal with camera specific features such as its focal length, skew, distortion, and image center. Extrinsic parameters describe its position and orientation in the world.  For the remainder of this page we will focus on estimating intrinsic parameters.
Camera calibration is the process of estimating the camera intrinsic and/or extrinsics parameters.  Intrinsic parameters deal with camera specific features such as its focal length, skew, distortion, and image center. Extrinsic parameters describe its position and orientation in the world.  For the remainder of this page we will focus on estimating intrinsic parameters.  Calibration is an essential first step for most 3D computer vision and removing lens distortion can often improve some times improve object detection accuracy.


BoofCV provides several software tools and support for different calibration target types.  In a typical scenario calibration will be done by viewing a calibration target from different locations and orientations.  Calibration points are then detected in these images.  Since the geometry of the calibration target is known the cameras intrinsic parameters can be estimated to a high level of accuracy.
BoofCV provides several software tools and support for different calibration target types.  In a typical scenario calibration will be done by viewing a calibration target from different locations and orientations.  Calibration points are then detected in these images.  Since the geometry of the calibration target is known the cameras intrinsic parameters can be estimated to a high level of accuracy.
Line 7: Line 7:
The most common way to calibrate a camera is using planar calibration targets with squares on them.  BoofCV provides support for two different types of planar calibration targets with different patterns of squares on them, chess board and square grid.  The code itself can be reused to detect similar types of application specific calibration targets.
The most common way to calibrate a camera is using planar calibration targets with squares on them.  BoofCV provides support for two different types of planar calibration targets with different patterns of squares on them, chess board and square grid.  The code itself can be reused to detect similar types of application specific calibration targets.


<center>'''CALIBRATION IS STILL A WORK IN PROGRESS'''</center>
<center>
Summary of current status:
<gallery caption="Different types of supported planar calibration grids" heights=150 widths=200 >
* Chessboard feature detector is accurate
File:Calib_target_chess_small.png|Chessboard pattern
* Square grid feature detector is decent but can be improved
File:Calib_target_square_small.png|Square grid pattern
* Numerical optimization needs to be improved but works OK
</gallery>
* API is lacking and there is no way to transfer found results to other parts of the code
</center>
 
 
= Quick Links =
 
Applets
* [[Applet_Calibrate_Planar_Mono| Calibrate Monocular Camera]]
* [[Applet_Calibrate_Planar_Stereo| Calibrate Stereo Camera]]
* [[Applet_Rectification_Calibrated| Calibrate Stereo Camera]]
 
Examples
* [[Example_Calibrate_Planar_Mono| Calibrate Monocular Camera]]
* [[Example_Calibrate_Planar_Stereo| Calibrate Stereo Camera]]
* [[Example_Remove_Lens_Distortion| Remove Lens Distortion]]
* [[Example_Rectification_Calibrated| Rectify Calibrated Stereo]]
* [[Example_Rectification_Uncalibrated| Rectify Uncalibrated Stereo]]
 
Calibration Targets:
 
* [https://github.com/lessthanoptimal/BoofCV-Data/blob/master/evaluation/calibration/letter_chess.ps| Letter Chessboard]
* [https://github.com/lessthanoptimal/BoofCV-Data/blob/master/evaluation/calibration/letter_square.ps| Letter Square Grid]
 
= Calibration Process =
 
In the following section the camera calibration procedure is broken down into steps and explained.  First a quick overview:
 
# Select a pattern, download, and print
# Mount the pattern onto a rigid flat surface
# Take pictures of the target using your camera
# Download pictures to compute and select the best ones
# Use provided examples to automatically detect calibration target and compute parameters
# Move outputted calibration file to a safe location
 
Which calibration target you use is a matter of preference.  Currently it is recommended that you use the chessboard pattern since it is producing more accurate results.  Other types of planar targets can be used by BoofCV if you provide the source code for detecting calibration points on the target.


Applet List
* [https://github.com/lessthanoptimal/BoofCV-Data/blob/master/evaluation/calibration/letter_chess.ps| Letter Chessboard]
* [[Applet_Calibrate_Detect_Square| Detect Square Targets]]
* [https://github.com/lessthanoptimal/BoofCV-Data/blob/master/evaluation/calibration/letter_square.ps| Letter Square Grid]
* [[Applet_Calibrate_Detect_Chess| Detect Chessboard Targets]]
* [[Applet_Calibrate_Zhang99| Calibrate Using Planar Target]]


While instructions are not yet provided, you can still view the code and figure out how to calibrate a camera using BoofCV.  Just look at the classes inside of the boofcv.alg.geo.calibration package
The target needs to be mounted on a surface as flat as reasonably possible.  Any warping of the paper will decrease its accuracy by introducing localization errors. A smooth table, glass, or any similar surface works well. Cardboard or foam is often warped or will warp as time goes on. Don't go overboard trying to make a perfectly smooth surface


== Why Work in Progress? ==
Taking a diverse set of sharp image is essential to calibration.  Images should be taken at several different orientations, distances, and locations in the image.  Be sure that the calibration target appears along the image edge and the center.  If all the pictures are taken in one region then the results will be biased, even if the residual error is low.  The whole target needs to be visible in the image and in some cases the target's border also needs to be visible.


Calibration is very much a work in progress because the API is still immature and the accuracy is a bit less than expected.  These preliminary results typically have a mean residual error between 0.2 and 0.5 pixels across the calibration image.  Which is within the typical range, but what makes the results some what suspect is how much variance there is between different datasets taken from the same camera.
Example code is provided for calibrating the cameras and applications which can visualize the results, see examples and evaluation directories.  If you use a calibration target that is different from the ones provided be sure to modify the example code with the correct target description. Look at calibration errors for individual images and see if there are any outliers or images in which the target was not detected.  Any targets with much larger errors should be removed since the image is probably bad.


=== Calibration with Square Grid ===
After the example code has run it will save the results into an XML file, see code.  Put this XML file is a safe location for future use in your project since.
To help validate its performance data provided on Zhengyou Zhang's [http://research.microsoft.com/en-us/um/people/zhang/Calib/#Calibration website] has been processed.  This test highlighted two issues.  1) The interest points provided by Zhang produced better results than the ones produced by BoofCV. 2) Even when using Zhang's interest points the found calibration parameters were similar, but not exactly the same.  [http://www.vision.caltech.edu/bouguetj/calib_doc/ Jean-Yves Bouguet's Matlab] calibration suite is able to produce nearly identical results to what Zhang had found.  In summary, what is currently in place now is a good start but has room for improvement in feature detection and parameter optimization.


=== Calibration with Chessboard ===
Example Code:
Detecting features in the chessboard target is an easier task.  The found accuracy is about 3x better (mean error of 0.1 pixels versus 0.3 pixels) than equivalent square grid datasets. 
# [[Example_Calibrate_Planar_Mono| Calibrate Monocular Camera]]
# [[Example_Calibrate_Planar_Stereo| Calibrate Stereo Camera]]


=== Numerical Optimization ===
= Lens Distortion =


Accuracy and speed of non-linear optimization can be improved.  An older variant of Levenberg-Marquardt (LM) is currently being used.  Plans are to use a trust region optimizer instead or a version of LM similar to what's in Minpack.  The Jacobian is currently computed numerically which will limit its accuracy and make it a bit slower.
Lens distortion can heavily distort features around the image border making them difficult to detect.  To overcome this problem the image can be undistorted, making as if an idealized camera is being used.  However this can be an expensive operation and some times feature are detected in the distorted image and their position correctly afterwards.


= Calibration Targets =
Example Code:
<center>
# [[Example_Remove_Lens_Distortion| Remove Lens Distortion]]
<gallery caption="Different types of supported planar calibration grids" heights=150 widths=200 >
 
File:Calib_target_chess_small.png|Chessboard pattern
= Stereo Rectification =
File:Calib_target_square_small.png|Square grid pattern
 
</gallery>
Stereo rectification is the process of distorting two images such that both their epipoles are at infinity, typically along the x-axis.  When this happens the epipolar lines are all parallel to each other simplifying the problem of finding feature correspondences to searching along the image axis.  Many stereo algorithms require images to be rectified first.
</center>
 
Rectification can be done on calibrated or uncalibrated images.  Calibration in this case refers to the stereo baseline (extrinsic parameters between two cameras) to be known. Although in practice it is often required that lens distortion be removed from the images even in the "uncalibrated" case.
 
The uncalibrated case can be done using automatically detected and associated features, however it is much tricker to get right than the calibrated case.  Any small association error will cause a large error in rectification.  Even if a state of the art and robust feature is used (e.g. SURF) and matches are pruned using the epipolar constraint, this alone will not be enough.  Additional knowledge of the scene needs to be taken in account.


Fully automatic algorithms are provided for detecting the two types of calibration grids above.  Calibration points are located at different locations on each target type.  For the chess board pattern only the interior corners are used while for the square grid pattern all corner points are used.
# [[Example_Rectification_Calibrated| Rectify Calibrated Stereo]]
# [[Example_Rectification_Uncalibrated| Rectify Uncalibrated Stereo]]

Revision as of 10:53, 22 April 2012

Camera Calibration

Camera calibration is the process of estimating the camera intrinsic and/or extrinsics parameters. Intrinsic parameters deal with camera specific features such as its focal length, skew, distortion, and image center. Extrinsic parameters describe its position and orientation in the world. For the remainder of this page we will focus on estimating intrinsic parameters. Calibration is an essential first step for most 3D computer vision and removing lens distortion can often improve some times improve object detection accuracy.

BoofCV provides several software tools and support for different calibration target types. In a typical scenario calibration will be done by viewing a calibration target from different locations and orientations. Calibration points are then detected in these images. Since the geometry of the calibration target is known the cameras intrinsic parameters can be estimated to a high level of accuracy.

The most common way to calibrate a camera is using planar calibration targets with squares on them. BoofCV provides support for two different types of planar calibration targets with different patterns of squares on them, chess board and square grid. The code itself can be reused to detect similar types of application specific calibration targets.

  • Different types of supported planar calibration grids
  • Calib target chess small.png

    Chessboard pattern

  • Calib target square small.png

    Square grid pattern


Quick Links

Applets

Examples

Calibration Targets:

Calibration Process

In the following section the camera calibration procedure is broken down into steps and explained. First a quick overview:

  1. Select a pattern, download, and print
  2. Mount the pattern onto a rigid flat surface
  3. Take pictures of the target using your camera
  4. Download pictures to compute and select the best ones
  5. Use provided examples to automatically detect calibration target and compute parameters
  6. Move outputted calibration file to a safe location

Which calibration target you use is a matter of preference. Currently it is recommended that you use the chessboard pattern since it is producing more accurate results. Other types of planar targets can be used by BoofCV if you provide the source code for detecting calibration points on the target.

The target needs to be mounted on a surface as flat as reasonably possible. Any warping of the paper will decrease its accuracy by introducing localization errors. A smooth table, glass, or any similar surface works well. Cardboard or foam is often warped or will warp as time goes on. Don't go overboard trying to make a perfectly smooth surface

Taking a diverse set of sharp image is essential to calibration. Images should be taken at several different orientations, distances, and locations in the image. Be sure that the calibration target appears along the image edge and the center. If all the pictures are taken in one region then the results will be biased, even if the residual error is low. The whole target needs to be visible in the image and in some cases the target's border also needs to be visible.

Example code is provided for calibrating the cameras and applications which can visualize the results, see examples and evaluation directories. If you use a calibration target that is different from the ones provided be sure to modify the example code with the correct target description. Look at calibration errors for individual images and see if there are any outliers or images in which the target was not detected. Any targets with much larger errors should be removed since the image is probably bad.

After the example code has run it will save the results into an XML file, see code. Put this XML file is a safe location for future use in your project since.

Example Code:

  1. Calibrate Monocular Camera
  2. Calibrate Stereo Camera

Lens Distortion

Lens distortion can heavily distort features around the image border making them difficult to detect. To overcome this problem the image can be undistorted, making as if an idealized camera is being used. However this can be an expensive operation and some times feature are detected in the distorted image and their position correctly afterwards.

Example Code:

  1. Remove Lens Distortion

Stereo Rectification

Stereo rectification is the process of distorting two images such that both their epipoles are at infinity, typically along the x-axis. When this happens the epipolar lines are all parallel to each other simplifying the problem of finding feature correspondences to searching along the image axis. Many stereo algorithms require images to be rectified first.

Rectification can be done on calibrated or uncalibrated images. Calibration in this case refers to the stereo baseline (extrinsic parameters between two cameras) to be known. Although in practice it is often required that lens distortion be removed from the images even in the "uncalibrated" case.

The uncalibrated case can be done using automatically detected and associated features, however it is much tricker to get right than the calibrated case. Any small association error will cause a large error in rectification. Even if a state of the art and robust feature is used (e.g. SURF) and matches are pruned using the epipolar constraint, this alone will not be enough. Additional knowledge of the scene needs to be taken in account.

  1. Rectify Calibrated Stereo
  2. Rectify Uncalibrated Stereo
Retrieved from "http://boofcv.org/index.php?title=Tutorial_Camera_Calibration&oldid=717"