Difference between revisions of "Example Rectification Calibrated"

From BoofCV
(Created page with "= Rectification of Calibrated Stereo = <center> <gallery heights=190 widths=550 > Image:Example_unrectified.jpg|Before rectification features are not aligned along the y-axis...")
 
(Updated for v0.8)
Line 8: Line 8:
 
</center>
 
</center>
  
Stereo rectification is the process of distorting an image such that the epipoles of both images are at infinity.  This as the effect that corresponding features must lie along the x-axis (or y-axis), thus enabling a quick search for features.  Many stereo vision algorithm rely on rectification.  The example below demonstrates rectification for a calibrated stereo pair.  Note that after calibration the new camera view has a different set of intrinsic parameters.
+
Stereo rectification is the process of distorting an image such that the epipoles of both images are at infinity.  If the epipoles are at infinity along the x-axis, then corresponding features must lie along the same y-coordinates.  Using knowledge that correspondence feature's have the same y-coordinate allows for quick searches.  Many stereo vision algorithm rely on rectification.  The example below demonstrates rectification for a calibrated stereo pair.  Note that after calibration the new camera view has a different set of intrinsic parameters.
  
 
Example File: [https://github.com/lessthanoptimal/BoofCV/blob/master/examples/src/boofcv/examples/ExampleRectifyCalibratedStereo.java ExampleRectifyCalibratedStereo.java]
 
Example File: [https://github.com/lessthanoptimal/BoofCV/blob/master/examples/src/boofcv/examples/ExampleRectifyCalibratedStereo.java ExampleRectifyCalibratedStereo.java]
Line 47: Line 47:
  
 
public static void main( String args[] ) {
 
public static void main( String args[] ) {
String dir = "../data/evaluation/calibration/stereo/Bumblebee2_Chess/";
+
String dir = "../data/applet/calibration/stereo/Bumblebee2_Chess/";
  
 
StereoParameters param = BoofMiscOps.loadXML(dir+"stereo.xml");
 
StereoParameters param = BoofMiscOps.loadXML(dir+"stereo.xml");
 
// Image coordinate are left handed.  This is the most common standard
 
// and is almost always true.
 
boolean leftHanded = true;
 
  
 
// load images
 
// load images
Line 74: Line 70:
  
 
// original camera calibration matrices
 
// original camera calibration matrices
DenseMatrix64F K1 = UtilIntrinsic.calibrationMatrix(param.getLeft());
+
DenseMatrix64F K1 = UtilIntrinsic.calibrationMatrix(param.getLeft(),null);
DenseMatrix64F K2 = UtilIntrinsic.calibrationMatrix(param.getRight());
+
DenseMatrix64F K2 = UtilIntrinsic.calibrationMatrix(param.getRight(),null);
  
 
rectifyAlg.process(K1,new Se3_F64(),K2,leftToRight);
 
rectifyAlg.process(K1,new Se3_F64(),K2,leftToRight);
Line 87: Line 83:
  
 
// Adjust the rectification to make the view area more useful
 
// Adjust the rectification to make the view area more useful
RectifyImageOps.fullViewLeft(param.left, leftHanded, rect1, rect2, rectK);
+
RectifyImageOps.fullViewLeft(param.left, rect1, rect2, rectK);
 
// RectifyImageOps.allInsideLeft(param.left, leftHanded, rect1, rect2, rectK);
 
// RectifyImageOps.allInsideLeft(param.left, leftHanded, rect1, rect2, rectK);
  
 
// undistorted and rectify images
 
// undistorted and rectify images
 
ImageDistort<ImageFloat32> imageDistortLeft =
 
ImageDistort<ImageFloat32> imageDistortLeft =
RectifyImageOps.rectifyImage(param.getLeft(), leftHanded, rect1, ImageFloat32.class);
+
RectifyImageOps.rectifyImage(param.getLeft(), rect1, ImageFloat32.class);
 
ImageDistort<ImageFloat32> imageDistortRight =
 
ImageDistort<ImageFloat32> imageDistortRight =
RectifyImageOps.rectifyImage(param.getRight(),leftHanded, rect2,ImageFloat32.class);
+
RectifyImageOps.rectifyImage(param.getRight(), rect2, ImageFloat32.class);
  
 
DistortImageOps.distortMS(distLeft, rectLeft, imageDistortLeft);
 
DistortImageOps.distortMS(distLeft, rectLeft, imageDistortLeft);

Revision as of 07:47, 12 May 2012

Rectification of Calibrated Stereo

Stereo rectification is the process of distorting an image such that the epipoles of both images are at infinity. If the epipoles are at infinity along the x-axis, then corresponding features must lie along the same y-coordinates. Using knowledge that correspondence feature's have the same y-coordinate allows for quick searches. Many stereo vision algorithm rely on rectification. The example below demonstrates rectification for a calibrated stereo pair. Note that after calibration the new camera view has a different set of intrinsic parameters.

Example File: ExampleRectifyCalibratedStereo.java

Concepts:

  • Stereo Rectification
  • Stereo Vision

Relevant Applets:

Related Examples:

Example Code

/**
 * <p>
 * Show how to rectify a pair of stereo images with known intrinsic parameters and stereo baseline.
 * The example code does the following:<br>
 * 1) Load stereo parameters from XML file with a pair of images.<br>
 * 2) Undistort and rectify images..  This provides one rectification matrix
 * for each image along with a new camera calibration matrix.<br>
 * 3) The original rectification does not try to maximize view area, however it can be adjusted.
 * 4)After rectification is finished the results are displayed.<br>
 * </p>
 *
 * <p>
 * Note that the y-axis in left and right images align after rectification.  The curved image edge
 * is an artifact of lens distortion being removed.
 * </p>
 *
 * @author Peter Abeles
 */
public class ExampleRectifyCalibratedStereo {

	public static void main( String args[] ) {
		String dir = "../data/applet/calibration/stereo/Bumblebee2_Chess/";

		StereoParameters param = BoofMiscOps.loadXML(dir+"stereo.xml");

		// load images
		BufferedImage origLeft = UtilImageIO.loadImage(dir+"left05.jpg");
		BufferedImage origRight = UtilImageIO.loadImage(dir+"right05.jpg");

		// distorted images
		MultiSpectral<ImageFloat32> distLeft = ConvertBufferedImage.convertFromMulti(origLeft, null, ImageFloat32.class);
		MultiSpectral<ImageFloat32> distRight = ConvertBufferedImage.convertFromMulti(origRight, null, ImageFloat32.class);

		// storage for undistorted + rectified images
		MultiSpectral<ImageFloat32> rectLeft = new MultiSpectral<ImageFloat32>(ImageFloat32.class,
				distLeft.getWidth(),distLeft.getHeight(),distLeft.getNumBands());
		MultiSpectral<ImageFloat32> rectRight = new MultiSpectral<ImageFloat32>(ImageFloat32.class,
				distRight.getWidth(),distRight.getHeight(),distRight.getNumBands());

		// Compute rectification
		RectifyCalibrated rectifyAlg = RectifyImageOps.createCalibrated();
		Se3_F64 leftToRight = param.getRightToLeft().invert(null);

		// original camera calibration matrices
		DenseMatrix64F K1 = UtilIntrinsic.calibrationMatrix(param.getLeft(),null);
		DenseMatrix64F K2 = UtilIntrinsic.calibrationMatrix(param.getRight(),null);

		rectifyAlg.process(K1,new Se3_F64(),K2,leftToRight);

		// rectification matrix for each image
		DenseMatrix64F rect1 = rectifyAlg.getRect1();
		DenseMatrix64F rect2 = rectifyAlg.getRect2();
		// New calibration matrix,
		// Both cameras have the same one after rectification.
		DenseMatrix64F rectK = rectifyAlg.getCalibrationMatrix();

		// Adjust the rectification to make the view area more useful
		RectifyImageOps.fullViewLeft(param.left, rect1, rect2, rectK);
//		RectifyImageOps.allInsideLeft(param.left, leftHanded, rect1, rect2, rectK);

		// undistorted and rectify images
		ImageDistort<ImageFloat32> imageDistortLeft =
				RectifyImageOps.rectifyImage(param.getLeft(), rect1, ImageFloat32.class);
		ImageDistort<ImageFloat32> imageDistortRight =
				RectifyImageOps.rectifyImage(param.getRight(), rect2, ImageFloat32.class);

		DistortImageOps.distortMS(distLeft, rectLeft, imageDistortLeft);
		DistortImageOps.distortMS(distRight, rectRight, imageDistortRight);

		// convert for output
		BufferedImage outLeft = ConvertBufferedImage.convertTo(rectLeft,null);
		BufferedImage outRight = ConvertBufferedImage.convertTo(rectRight, null);

		// show results and draw a horizontal line where the user clicks to see rectification easier
		ListDisplayPanel panel = new ListDisplayPanel();
		panel.addItem(new RectifiedPairPanel(true, origLeft, origRight), "Original");
		panel.addItem(new RectifiedPairPanel(true, outLeft, outRight), "Rectified");

		ShowImages.showWindow(panel,"Stereo Rectification Calibrated");
	}
}