Difference between revisions of "Example Remove Lens Distortion"

From BoofCV
Jump to navigationJump to search
m
m
(16 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Removing Lens Distortion =
<center>
<center>
<gallery caption="Images with lens distortion removed" heights=240 widths=320 >
<gallery caption="Images with lens distortion removed" heights=240 widths=320 >
Line 8: Line 6:
</center>
</center>


After a camera has been calibrated its intrinsic parameters and lens distortion parameters are known.  This example demonstrates how to load those parameters from an XML file and remove lens distortion from an image.  After lens distortion is removed, the new undistorted view often pushes pixels outside the viewing area.  To compensate for this problem it is possible to adjust the view to maximize the view area based on different objectives.  After adjusting the view be sure to use the new intrinsic parameters when performing geometric operations.
This example demonstrates how to load a camera's calibration from an XML file and remove lens distortion from its images.  After lens distortion is removed, the new undistorted view often "pushes" pixels outside the viewing area.  To compensate for this problem it is possible to adjust the view to maximize the view area based on different objectives.  After adjusting the view be sure to use the new intrinsic parameters when performing geometric operations.


Example File: [https://github.com/lessthanoptimal/BoofCV/blob/master/examples/src/boofcv/examples/ExampleRemoveLensDistortion.java ExampleRemoveLensDistortion.java]
Example File: [https://github.com/lessthanoptimal/BoofCV/blob/v0.37/examples/src/boofcv/examples/calibration/ExampleRemoveLensDistortion.java ExampleRemoveLensDistortion.java]


Concepts:
Concepts:
Line 16: Line 14:
* Calibration
* Calibration
* Intrinsic Parameters
* Intrinsic Parameters
Relevant Applets:
* [[Applet_Remove_Lens_Distortion| Removing Lens Distortion]]
* [[Applet_Lens_Distortion| Demonstration of Lens Distortion Parameters]]
* [[Applet_Calibrate_Planar_Mono| Monocular Camera Calibration]]


Related Examples:
Related Examples:
Line 33: Line 26:
  *
  *
  * After lens distortion has been removed the new image will not be properly contained inside the original
  * After lens distortion has been removed the new image will not be properly contained inside the original
  * image side.  Several methods are provided for scaling and translating the image to maximize the view area
  * image side.  Methods are provided for scaling and translating the image to maximize the view area
  * using different metrics.  After this adjustment has been done the new image is equivalent to one being
  * using different metrics.  After this adjustment has been done the new image is equivalent to one being
  * generated by a virtual camera with a different set of intrinsic parameters.
  * generated by a virtual camera with a different set of intrinsic parameters.
Line 41: Line 34:
public class ExampleRemoveLensDistortion {
public class ExampleRemoveLensDistortion {


// Assume image coordinate are left handed.  This is the most common standard and is almost always true.
public static void main( String[] args ) {
private static boolean leftHanded = true;
String calibDir = UtilIO.pathExample("calibration/mono/Sony_DSC-HX5V_Chess/");
 
String imageDir = UtilIO.pathExample("structure/");
public static void main( String args[] ) {
String calibDir = "../data/evaluation/calibration/mono/Sony_DSC-HX5V_Chess/";
String imageDir = "../data/evaluation/structure/";


// load calibration parameters from the previously calibrated camera
// load calibration parameters from the previously calibrated camera
IntrinsicParameters param = BoofMiscOps.loadXML(calibDir + "intrinsic.xml");
CameraPinholeBrown param = CalibrationIO.load(new File(calibDir , "intrinsic.yaml"));
// Specify a transform that has no lens distortion that you wish to re-render the image as having
CameraPinhole desired = new CameraPinhole(param);


// load images and convert the image into a color BoofCV format
// load images and convert the image into a color BoofCV format
BufferedImage orig = UtilImageIO.loadImage(imageDir + "dist_cyto_01.jpg");
BufferedImage orig = UtilImageIO.loadImage(imageDir , "dist_cyto_01.jpg");
MultiSpectral<ImageFloat32> distortedImg = ConvertBufferedImage.convertFromMulti(orig, null, ImageFloat32.class);
Planar<GrayF32> distortedImg =
ConvertBufferedImage.convertFromPlanar(orig, null,true, GrayF32.class);


// compute the transform to remove lens distortion
int numBands = distortedImg.getNumBands();
// The inverse transformation (adds distortion) is used when apply adjusting an image.
// In other application the forward transformation (removes distortion) is required.
PointTransform_F32 tran = LensDistortionOps.radialTransformInv(param, leftHanded);


// create new transforms to optimize view area
// create new transforms which optimize view area in different ways.
// After distortion the adjusted intrinsic camera parameters should be used.
// EXPAND makes sure there are no black outside of image pixels inside the image
// Since they are not being used in this example null is passed in.
// FULL_VIEW will include the entire original image
PointTransform_F32 fullView = LensDistortionOps.fullView(param, leftHanded, null);
// The border is VALUE, which defaults to black, just so you can see it
PointTransform_F32 allInside = LensDistortionOps.allInside(param, leftHanded, null);
ImageDistort allInside = LensDistortionOps.changeCameraModel(AdjustmentType.EXPAND, BorderType.ZERO,
param, desired,null, ImageType.pl(numBands, GrayF32.class));
ImageDistort fullView = LensDistortionOps.changeCameraModel(AdjustmentType.FULL_VIEW, BorderType.ZERO,
param, desired, null, ImageType.pl(numBands, GrayF32.class));


// Set up image distort
// NOTE: After lens distortion has been removed the intrinsic parameters is changed.  If you pass
InterpolatePixel<ImageFloat32> interp = FactoryInterpolation.bilinearPixel(ImageFloat32.class);
//      in  a set of IntrinsicParameters to the 4th variable it will save it there.
ImageDistort<ImageFloat32> distort = FactoryDistort.distort(interp,null,ImageFloat32.class);
// NOTE: Type information was stripped from ImageDistort simply because it becomes too verbose with it here.
//      Would be nice if this verbosity issue was addressed by the Java language.


// render and display the different types of views in a window
// render and display the different types of views in a window
displayResults(orig, distortedImg, tran, fullView, allInside, distort);
displayResults(orig, distortedImg, allInside, fullView );
}
}


Line 78: Line 72:
*/
*/
private static void displayResults(BufferedImage orig,
private static void displayResults(BufferedImage orig,
  MultiSpectral<ImageFloat32> distortedImg,
  Planar<GrayF32> distortedImg,
  PointTransform_F32 tran,
  ImageDistort allInside, ImageDistort fullView ) {
  PointTransform_F32 fullView,
  PointTransform_F32 allInside,
  ImageDistort<ImageFloat32> distort) {
// render the results
// render the results
MultiSpectral<ImageFloat32> undistortedImg = new MultiSpectral<ImageFloat32>(ImageFloat32.class,
Planar<GrayF32> undistortedImg = new Planar<>(GrayF32.class,
distortedImg.getWidth(),distortedImg.getHeight(),distortedImg.getNumBands());
distortedImg.getWidth(),distortedImg.getHeight(),distortedImg.getNumBands());


distort.setModel(new PointToPixelTransform_F32(tran));
allInside.apply(distortedImg, undistortedImg);
GeneralizedImageOps.fill(undistortedImg, 0);
BufferedImage out1 = ConvertBufferedImage.convertTo(undistortedImg, null,true);
DistortImageOps.distortMS(distortedImg, undistortedImg, distort);
BufferedImage out1 = ConvertBufferedImage.convertTo(undistortedImg, null);
 
distort.setModel(new PointToPixelTransform_F32(fullView));
GeneralizedImageOps.fill(undistortedImg,0);
DistortImageOps.distortMS(distortedImg,undistortedImg,distort);
BufferedImage out2 = ConvertBufferedImage.convertTo(undistortedImg,null);


distort.setModel(new PointToPixelTransform_F32(allInside));
fullView.apply(distortedImg,undistortedImg);
GeneralizedImageOps.fill(undistortedImg,0);
BufferedImage out2 = ConvertBufferedImage.convertTo(undistortedImg, null,true);
DistortImageOps.distortMS(distortedImg,undistortedImg,distort);
BufferedImage out3 = ConvertBufferedImage.convertTo(undistortedImg,null);


// display in a single window where the user can easily switch between images
// display in a single window where the user can easily switch between images
ListDisplayPanel panel = new ListDisplayPanel();
ListDisplayPanel panel = new ListDisplayPanel();
panel.addItem(new ImagePanel(orig), "Original");
panel.addItem(new ImagePanel(orig), "Original");
panel.addItem(new ImagePanel(out1), "Undistorted");
panel.addItem(new ImagePanel(out1), "Undistorted All Inside");
panel.addItem(new ImagePanel(out3), "Undistorted All Inside");
panel.addItem(new ImagePanel(out2), "Undistorted Full View");
panel.addItem(new ImagePanel(out2), "Undistorted Full View");


ShowImages.showWindow(panel, "Removing Lens Distortion");
ShowImages.showWindow(panel, "Removing Lens Distortion", true);
}
}
}
}
</syntaxhighlight>
</syntaxhighlight>

Revision as of 12:48, 21 December 2020

This example demonstrates how to load a camera's calibration from an XML file and remove lens distortion from its images. After lens distortion is removed, the new undistorted view often "pushes" pixels outside the viewing area. To compensate for this problem it is possible to adjust the view to maximize the view area based on different objectives. After adjusting the view be sure to use the new intrinsic parameters when performing geometric operations.

Example File: ExampleRemoveLensDistortion.java

Concepts:

  • Lens Distortion
  • Calibration
  • Intrinsic Parameters

Related Examples:

Example Code

/**
 * All real camera lens have distortion.  This distortion causes large errors when attempting to recover the
 * scene's structure and camera's motion.  The following example demonstrates how the lens distortion can be
 * removed from an image after the camera has been calibrated.
 *
 * After lens distortion has been removed the new image will not be properly contained inside the original
 * image side.  Methods are provided for scaling and translating the image to maximize the view area
 * using different metrics.  After this adjustment has been done the new image is equivalent to one being
 * generated by a virtual camera with a different set of intrinsic parameters.
 *
 * @author Peter Abeles
 */
public class ExampleRemoveLensDistortion {

	public static void main( String[] args ) {
		String calibDir = UtilIO.pathExample("calibration/mono/Sony_DSC-HX5V_Chess/");
		String imageDir = UtilIO.pathExample("structure/");

		// load calibration parameters from the previously calibrated camera
		CameraPinholeBrown param = CalibrationIO.load(new File(calibDir , "intrinsic.yaml"));
		// Specify a transform that has no lens distortion that you wish to re-render the image as having
		CameraPinhole desired = new CameraPinhole(param);

		// load images and convert the image into a color BoofCV format
		BufferedImage orig = UtilImageIO.loadImage(imageDir , "dist_cyto_01.jpg");
		Planar<GrayF32> distortedImg =
				ConvertBufferedImage.convertFromPlanar(orig, null,true, GrayF32.class);

		int numBands = distortedImg.getNumBands();

		// create new transforms which optimize view area in different ways.
		// EXPAND makes sure there are no black outside of image pixels inside the image
		// FULL_VIEW will include the entire original image
		// The border is VALUE, which defaults to black, just so you can see it
		ImageDistort allInside = LensDistortionOps.changeCameraModel(AdjustmentType.EXPAND, BorderType.ZERO,
				param, desired,null, ImageType.pl(numBands, GrayF32.class));
		ImageDistort fullView = LensDistortionOps.changeCameraModel(AdjustmentType.FULL_VIEW, BorderType.ZERO,
				param, desired, null, ImageType.pl(numBands, GrayF32.class));

		// NOTE: After lens distortion has been removed the intrinsic parameters is changed.  If you pass
		//       in  a set of IntrinsicParameters to the 4th variable it will save it there.
		// NOTE: Type information was stripped from ImageDistort simply because it becomes too verbose with it here.
		//       Would be nice if this verbosity issue was addressed by the Java language.

		// render and display the different types of views in a window
		displayResults(orig, distortedImg, allInside, fullView );
	}

	/**
	 * Displays results in a window for easy comparison..
	 */
	private static void displayResults(BufferedImage orig,
									   Planar<GrayF32> distortedImg,
									   ImageDistort allInside, ImageDistort fullView ) {
		// render the results
		Planar<GrayF32> undistortedImg = new Planar<>(GrayF32.class,
				distortedImg.getWidth(),distortedImg.getHeight(),distortedImg.getNumBands());

		allInside.apply(distortedImg, undistortedImg);
		BufferedImage out1 = ConvertBufferedImage.convertTo(undistortedImg, null,true);

		fullView.apply(distortedImg,undistortedImg);
		BufferedImage out2 = ConvertBufferedImage.convertTo(undistortedImg, null,true);

		// display in a single window where the user can easily switch between images
		ListDisplayPanel panel = new ListDisplayPanel();
		panel.addItem(new ImagePanel(orig), "Original");
		panel.addItem(new ImagePanel(out1), "Undistorted All Inside");
		panel.addItem(new ImagePanel(out2), "Undistorted Full View");

		ShowImages.showWindow(panel, "Removing Lens Distortion", true);
	}
}