Example Remove Lens Distortion

• Images with lens distortion removed
• 

  Adjusted to maximize the view without including "black" pixels.

• 

  Adjusted so that the entire original image can be viewed.

This example demonstrates how to load a camera's calibration from an XML file and remove lens distortion from its images. After lens distortion is removed, the new undistorted view often "pushes" pixels outside the viewing area. To compensate for this problem it is possible to adjust the view to maximize the view area based on different objectives. After adjusting the view be sure to use the new intrinsic parameters when performing geometric operations.

Example File: ExampleRemoveLensDistortion.java

Concepts:

• Lens Distortion
• Calibration
• Intrinsic Parameters

Related Examples:

• Monocular Camera Calibration

Example Code

/**
 * All real camera lens have distortion. This distortion causes large errors when attempting to recover the
 * scene's structure and camera's motion. The following example demonstrates how the lens distortion can be
 * removed from an image after the camera has been calibrated.
 *
 * After lens distortion has been removed the new image will not be properly contained inside the original
 * image side. Methods are provided for scaling and translating the image to maximize the view area
 * using different metrics. After this adjustment has been done the new image is equivalent to one being
 * generated by a virtual camera with a different set of intrinsic parameters.
 *
 * @author Peter Abeles
 */
public class ExampleRemoveLensDistortion {

	public static void main( String[] args ) {
		String calibDir = UtilIO.pathExample("calibration/mono/Sony_DSC-HX5V_Chess/");
		String imageDir = UtilIO.pathExample("structure/");

		// load calibration parameters from the previously calibrated camera
		CameraPinholeBrown param = CalibrationIO.load(new File(calibDir , "intrinsic.yaml"));
		// Specify a transform that has no lens distortion that you wish to re-render the image as having
		CameraPinhole desired = new CameraPinhole(param);

		// load images and convert the image into a color BoofCV format
		BufferedImage orig = UtilImageIO.loadImage(imageDir , "dist_cyto_01.jpg");
		Planar<GrayF32> distortedImg =
				ConvertBufferedImage.convertFromPlanar(orig, null,true, GrayF32.class);

		int numBands = distortedImg.getNumBands();

		// create new transforms which optimize view area in different ways.
		// EXPAND makes sure there are no black outside of image pixels inside the image
		// FULL_VIEW will include the entire original image
		// The border is VALUE, which defaults to black, just so you can see it
		ImageDistort allInside = LensDistortionOps.changeCameraModel(AdjustmentType.EXPAND, BorderType.ZERO,
				param, desired,null, ImageType.pl(numBands, GrayF32.class));
		ImageDistort fullView = LensDistortionOps.changeCameraModel(AdjustmentType.FULL_VIEW, BorderType.ZERO,
				param, desired, null, ImageType.pl(numBands, GrayF32.class));

		// NOTE: After lens distortion has been removed the intrinsic parameters is changed. If you pass
		//       in  a set of IntrinsicParameters to the 4th variable it will save it there.
		// NOTE: Type information was stripped from ImageDistort simply because it becomes too verbose with it here.
		//       Would be nice if this verbosity issue was addressed by the Java language.

		// render and display the different types of views in a window
		displayResults(orig, distortedImg, allInside, fullView );
	}

	/**
	 * Displays results in a window for easy comparison..
	 */
	private static void displayResults(BufferedImage orig,
									   Planar<GrayF32> distortedImg,
									   ImageDistort allInside, ImageDistort fullView ) {
		// render the results
		Planar<GrayF32> undistortedImg = new Planar<>(GrayF32.class,
				distortedImg.getWidth(),distortedImg.getHeight(),distortedImg.getNumBands());

		allInside.apply(distortedImg, undistortedImg);
		BufferedImage out1 = ConvertBufferedImage.convertTo(undistortedImg, null,true);

		fullView.apply(distortedImg,undistortedImg);
		BufferedImage out2 = ConvertBufferedImage.convertTo(undistortedImg, null,true);

		// display in a single window where the user can easily switch between images
		ListDisplayPanel panel = new ListDisplayPanel();
		panel.addItem(new ImagePanel(orig), "Original");
		panel.addItem(new ImagePanel(out1), "Undistorted All Inside");
		panel.addItem(new ImagePanel(out2), "Undistorted Full View");

		ShowImages.showWindow(panel, "Removing Lens Distortion", true);
	}
}