Example Remove Lens Distortion

Rectification of Calibrated Stereo

• Images with lens distortion removed
• 

  Adjusted to maximize the view without including "black" pixels.

• 

  Adjusted so that the entire original image can be viewed.

After a camera has been calibrated its intrinsic parameters and lens distortion parameters are known. This example demonstrates how to load those parameters from an XML file and remove lens distortion from an image. After lens distortion is removed, the new undistorted view often pushes pixels outside the viewing area. To compensate for this problem it is possible to adjust the view to maximize the view area based on different objectives. After adjusting the view be sure to use the new intrinsic parameters when performing geometric operations.

Example File: ExampleRemoveLensDistortion.java

Concepts:

• Lens Distortion
• Calibration
• Intrinsic Parameters

Relevant Applets:

• Demonstration of Lens Distortion
• Monocular Camera Calibration

Related Examples:

• Monocular Camera Calibration

Example Code

/**
 * All real camera lens have distortion.  This distortion causes large errors when attempting to recover the
 * scene's structure and camera's motion.  The following example demonstrates how the lens distortion can be
 * removed from an image after the camera has been calibrated.
 *
 * After lens distortion has been removed the new image will not be properly contained inside the original
 * image side.  Several methods are provided for scaling and translating the image to maximize the view area
 * using different metrics.  After this adjustment has been done the new image is equivalent to one being
 * generated by a virtual camera with a different set of intrinsic parameters.
 *
 * @author Peter Abeles
 */
public class ExampleRemoveLensDistortion {

	// Assume image coordinate are left handed.  This is the most common standard and is almost always true.
	private static boolean leftHanded = true;

	public static void main( String args[] ) {
		String calibDir = "../data/evaluation/calibration/mono/Sony_DSC-HX5V_Chess/";
		String imageDir = "../data/evaluation/structure/";

		// load calibration parameters from the previously calibrated camera
		IntrinsicParameters param = BoofMiscOps.loadXML(calibDir + "intrinsic.xml");

		// load images and convert the image into a color BoofCV format
		BufferedImage orig = UtilImageIO.loadImage(imageDir + "dist_cyto_01.jpg");
		MultiSpectral<ImageFloat32> distortedImg = ConvertBufferedImage.convertFromMulti(orig, null, ImageFloat32.class);

		// compute the transform to remove lens distortion
		// The inverse transformation (adds distortion) is used when apply adjusting an image.
		// In other application the forward transformation (removes distortion) is required.
		PointTransform_F32 tran = LensDistortionOps.radialTransformInv(param, leftHanded);

		// create new transforms to optimize view area
		// After distortion the adjusted intrinsic camera parameters should be used.
		// Since they are not being used in this example null is passed in.
		PointTransform_F32 fullView = LensDistortionOps.fullView(param, leftHanded, null);
		PointTransform_F32 allInside = LensDistortionOps.allInside(param, leftHanded, null);

		// Set up image distort
		InterpolatePixel<ImageFloat32> interp = FactoryInterpolation.bilinearPixel(ImageFloat32.class);
		ImageDistort<ImageFloat32> distort = FactoryDistort.distort(interp,null,ImageFloat32.class);

		// render and display the different types of views in a window
		displayResults(orig, distortedImg, tran, fullView, allInside, distort);
	}

	/**
	 * Displays results in a window for easy comparison..
	 */
	private static void displayResults(BufferedImage orig,
									   MultiSpectral<ImageFloat32> distortedImg,
									   PointTransform_F32 tran,
									   PointTransform_F32 fullView,
									   PointTransform_F32 allInside,
									   ImageDistort<ImageFloat32> distort) {
		// render the results
		MultiSpectral<ImageFloat32> undistortedImg = new MultiSpectral<ImageFloat32>(ImageFloat32.class,
				distortedImg.getWidth(),distortedImg.getHeight(),distortedImg.getNumBands());

		distort.setModel(new PointToPixelTransform_F32(tran));
		GeneralizedImageOps.fill(undistortedImg, 0);
		DistortImageOps.distortMS(distortedImg, undistortedImg, distort);
		BufferedImage out1 = ConvertBufferedImage.convertTo(undistortedImg, null);

		distort.setModel(new PointToPixelTransform_F32(fullView));
		GeneralizedImageOps.fill(undistortedImg,0);
		DistortImageOps.distortMS(distortedImg,undistortedImg,distort);
		BufferedImage out2 = ConvertBufferedImage.convertTo(undistortedImg,null);

		distort.setModel(new PointToPixelTransform_F32(allInside));
		GeneralizedImageOps.fill(undistortedImg,0);
		DistortImageOps.distortMS(distortedImg,undistortedImg,distort);
		BufferedImage out3 = ConvertBufferedImage.convertTo(undistortedImg,null);

		// display in a single window where the user can easily switch between images
		ListDisplayPanel panel = new ListDisplayPanel();
		panel.addItem(new ImagePanel(orig), "Original");
		panel.addItem(new ImagePanel(out1), "Undistorted");
		panel.addItem(new ImagePanel(out3), "Undistorted All Inside");
		panel.addItem(new ImagePanel(out2), "Undistorted Full View");

		ShowImages.showWindow(panel, "Removing Lens Distortion");
	}
}