Difference between revisions of "Example Image Filter"

From BoofCV
Jump to navigationJump to search
m
m
(18 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Introduction =
<center>
<center>
<gallery caption="Sequence of processing steps in example." heights=200 widths=200 >
<gallery caption="Sequence of processing steps in example." heights=200 widths=200 >
Image:lenna_gray.jpg|Input: [http://en.wikipedia.org/wiki/Lenna Lenna] input Image.
File:Kodim17_face_orig.jpg|Input Image.
Image:example_lenna_gaussianblur.jpg|Gaussian blur.
File:Kodim17_face_gaussian.jpg|Gaussian blur.
Image:example_image_filter.jpg|Output: X-axis derivative.
File:Kodim17_gradient.jpg|Colorized Gradient.
</gallery>
</gallery>
</center>
</center>
Line 15: Line 13:
Inside the children of 'boofcv.abst' package are abstracted filters.  These classes provide an Object Oriented Programming (OOP) interface and handle much of the memory management, but are less flexible.  Some times the ability to configure low level parameters has been removed from the algorithms that they are wrappers around to make development more manageable.
Inside the children of 'boofcv.abst' package are abstracted filters.  These classes provide an Object Oriented Programming (OOP) interface and handle much of the memory management, but are less flexible.  Some times the ability to configure low level parameters has been removed from the algorithms that they are wrappers around to make development more manageable.


Example File: [https://github.com/lessthanoptimal/BoofCV/blob/master/examples/src/boofcv/examples/ImageFilterExample.java ImageFilterExample]
Example File: [https://github.com/lessthanoptimal/BoofCV/blob/v0.33/examples/src/main/java/boofcv/examples/imageprocessing/ExampleImageFilter.java ExampleImageFilter]


Tutorial Concepts:
Tutorial Concepts:
Line 24: Line 22:
# Display images.
# Display images.


Relevant Applets:
Relevant Examples:
# [[Applet_Image_Blur | Image Blur]]
# [[Example_Image_Blur | Image Blur]]
# [[Applet_Gradient | Gradient]]
# [[Example_Image_Derivative | Image Derivative]]


= Procedural =
= Procedural =


Here an ImageUInt8 image is passed in, which is then blurred, has its derivative computed and displayed in a window.  In BoofCV, unlike many other vision libraries, to provide strong type checking at compile time, images with different primitive data types are a different type.  The name 'UInt8' indicates that its elements are unsigned 8-bit integers and that it has only a single band.  Image derivative require a different data type because the domain of each element includes negative numbers and has a larger domain, which is why they are stored in a ImageSInt16ImageSInt16 are single band images with signed 16-bit integer pixels.   
Here an GrayU8 image is passed in, which is then blurred, has its derivative computed and displayed in a window.  In BoofCV, unlike many other vision libraries, to provide strong type checking at compile time, images with different primitive data types are a different type.  The name 'U8' indicates that its elements are unsigned 8-bit integers and that it has only a single band.  Image derivative require a different data type because the domain of each element includes negative numbers and has a larger domain, which is why they are stored in a GrayS16GrayS16 are single band images with signed 16-bit integer pixels.   


<syntaxhighlight lang="java">
<syntaxhighlight lang="java">
public static void procedural( ImageUInt8 input )
public static void procedural( GrayU8 input )
{
{
ImageUInt8 blurred = new ImageUInt8(input.width,input.height);
GrayU8 blurred = new GrayU8(input.width,input.height);
ImageSInt16 derivX = new ImageSInt16(input.width,input.height);
GrayS16 derivX = new GrayS16(input.width,input.height);
ImageSInt16 derivY = new ImageSInt16(input.width,input.height);
GrayS16 derivY = new GrayS16(input.width,input.height);


// Gaussian blur: Convolve a Gaussian kernel with a width of 5 pixels
// Gaussian blur: Convolve a Gaussian kernel
BlurImageOps.gaussian(input,blurred,-1,2,null);
BlurImageOps.gaussian(input,blurred,-1,blurRadius,null);


// Calculate image's derivative
// Calculate image's derivative
GradientSobel.process(blurred, derivX, derivY, FactoryImageBorder.extend(input));
GradientSobel.process(blurred, derivX, derivY, FactoryImageBorderAlgs.extend(input));


// display the results
// display the results
BufferedImage outputImage = VisualizeImageData.colorizeSign(derivX,null,-1);
BufferedImage outputImage = VisualizeImageData.colorizeGradient(derivX, derivY, -1, null);
ShowImages.showWindow(outputImage,"Procedural Fixed Type");
panel.addImage(outputImage,"Procedural Fixed Type");
}
}
</syntaxhighlight>
</syntaxhighlight>


Notice how in the gaussian() function (-1,2) are passed inThe negative number indicates that the procedure should select the Gaussian's sigma itself using the provided kernel radius of 2A null is also passed into that function. When computing the Gaussian blur a temporary image is required, since null was passed in for the last parameter it will internally declare that memory.  Declaring temporary memory each time a function is invoked is wasteful, but easier to program.
The two integer parameters in BlurImageOps.gaussian(input,blurred,-1,2,null) specify the blur's sigma and radiusA negative sigma indicates that the sigma should be chosen based upon the kernel's radius.  The converse is also true, a negative radius means the radius should be selected based upon the sigma's value. When applying Gaussian blur, additional storage is required. Because null was passed in for the last parameter, it will internally declare that memory.  Declaring temporary memory each time a function is invoked is wasteful, but easier to program.


When convolving a kernel across an image its not always clear how image borders should be handled where only part of the image intersects the kernel. In this example, when computing the gradient using a Sobel operator, it is creating a function that will handle the border by extending the outside pixels.
When convolving a kernel across an image its not always clear how image borders should be handled.  At the image border convolution kernels will be partially outside of the image. In this example, the gradient is computed using a [http://en.wikipedia.org/wiki/Sobel_operator Sobel kernel], and BorderType.EXTENDED tells it to handle the border by extending border pixels to outside the image.


Finally the output is computed by rendering the image derivative into a BufferedImage.  [http://download.oracle.com/javase/6/docs/api/java/awt/image/BufferedImage.html BufferedImages] come from [http://download.oracle.com/javase/tutorial/uiswing/start/index.html Java Swing] and is Java's standard way for storing image data.  They are not used internally in BoofCV due to their complexity and poor performance.  Once rendered, it is displayed in an image using the convenient showWindow() function.
Finally the output is computed by rendering the image derivative into a BufferedImage.  [http://download.oracle.com/javase/8/docs/api/java/awt/image/BufferedImage.html BufferedImages] come from [http://download.oracle.com/javase/tutorial/uiswing/start/index.html Java Swing] and is Java's standard way for storing image data.  They are not used internally in BoofCV due to their complexity and poor performance.  Once rendered, it is displayed in an image using the convenient showWindow() function.


= Generalized =
= Generalized =


The following example is very similar to the first, but it now takes in a generalized image type.  Type checking is weaker, but many different images can be passed in now.   
The following example is very similar to the first, but it now takes in a generalized image type using [http://download.oracle.com/javase/tutorial/java/generics/index.html Java generics].  Type checking is weaker, but different image types can be passed in using the exact same code.   


<syntaxhighlight lang="java">
<syntaxhighlight lang="java">
public static <T extends ImageBase, D extends ImageBase>
public static <T extends ImageGray<T>, D extends ImageGray<D>>
void generalized( T input )
void generalized( T input )
{
{
Line 68: Line 66:
Class<D> derivType = GImageDerivativeOps.getDerivativeType(inputType);
Class<D> derivType = GImageDerivativeOps.getDerivativeType(inputType);


T blurred = GeneralizedImageOps.createImage(inputType,input.width, input.height);
T blurred = (T)input.createSameShape();
D derivX = GeneralizedImageOps.createImage(derivType,input.width, input.height);
D derivX = GeneralizedImageOps.createSingleBand(derivType, input.width, input.height);
D derivY = GeneralizedImageOps.createImage(derivType,input.width, input.height);
D derivY = GeneralizedImageOps.createSingleBand(derivType, input.width, input.height);


// Gaussian blur: Convolve a Gaussian kernel with a width of 5 pixels
// Gaussian blur: Convolve a Gaussian kernel
GBlurImageOps.gaussian(input, blurred, -1, 2, null);
GBlurImageOps.gaussian(input, blurred, -1, blurRadius, null);


// Calculate image's derivative
// Calculate image's derivative
GImageDerivativeOps.sobel(blurred, derivX, derivY, BorderType.EXTENDED);
GImageDerivativeOps.gradient(DerivativeType.SOBEL,blurred, derivX, derivY, BorderType.EXTENDED);


// display the results
// display the results
BufferedImage outputImage = VisualizeImageData.colorizeSign(derivX,null,-1);
BufferedImage outputImage = VisualizeImageData.colorizeGradient(derivX, derivY,-1, null);
ShowImages.showWindow(outputImage,"Generalized "+inputType.getSimpleName());
panel.addImage(outputImage,"Generalized "+inputType.getSimpleName());
}
}
</syntaxhighlight>
</syntaxhighlight>


Temporary images are created using GeneralizedImageOps.createImage() sicne the type is not known at compile timeEven though the specific type of image isn't know at runtime, Java generics allow consistency checks to make sure images which should be the same type are the same type.  Notice that blurring and image derivatives are computed using classes that begin with the letter 'G', which signifies they are generalized.
Since the type is not known at compile type, images are created using GeneralizedImageOps.createImage().  Java generics allow consistency checks to make sure images which should be the same type are the same type.  Notice that blurring and image derivatives are computed using classes that begin with the letter 'G', that signifies they are generalized classes.


= Abstracted =
= Abstracted =
Line 91: Line 89:


<syntaxhighlight lang="java">
<syntaxhighlight lang="java">
public static <T extends ImageBase, D extends ImageBase>
public static <T extends ImageGray<T>, D extends ImageGray<D>>
void filter( T input )
void filter( T input )
{
{
Line 97: Line 95:
Class<D> derivType = GImageDerivativeOps.getDerivativeType(inputType);
Class<D> derivType = GImageDerivativeOps.getDerivativeType(inputType);


T blurred = GeneralizedImageOps.createImage(inputType, input.width, input.height);
T blurred = (T)input.createSameShape();
D derivX = GeneralizedImageOps.createImage(derivType, input.width, input.height);
D derivX = GeneralizedImageOps.createSingleBand(derivType, input.width, input.height);
D derivY = GeneralizedImageOps.createImage(derivType, input.width, input.height);
D derivY = GeneralizedImageOps.createSingleBand(derivType, input.width, input.height);


// declare image filters
// declare image filters
BlurFilter<T> filterBlur = FactoryBlurFilter.gaussian(inputType, -1, 2);
BlurFilter<T> filterBlur = FactoryBlurFilter.gaussian(ImageType.single(inputType), -1, blurRadius);
ImageGradient<T,D> gradient = FactoryDerivative.sobel(inputType, derivType);
ImageGradient<T,D> gradient = FactoryDerivative.sobel(inputType, derivType);


Line 110: Line 108:


// display the results
// display the results
BufferedImage outputImage = VisualizeImageData.colorizeSign(derivX,null,-1);
BufferedImage outputImage = VisualizeImageData.colorizeGradient(derivX, derivY, -1, null);
ShowImages.showWindow(outputImage,"Filter "+inputType.getSimpleName());
panel.addImage(outputImage,"Filter "+inputType.getSimpleName());
}
}
</syntaxhighlight>
</syntaxhighlight>


Notice how the filters are created using a Factory class. Factories provide an easy to use interface for constructing filters and other algorithms.  Often they will handle all the little details and only let the user specify major parameters.  Notice how FactoryDerivative.sobel() does not allow the user to specify the border type, that's because it assumes it knows the best type and might as well spare the developer from that level of detail.
Notice how Factories are used to create filters. Factories provide an easy to use interface for constructing filters and other algorithms.  They also provide a single location to look for similar algorithms which implement the same interface.  Often they will simplify development by handling all the little details and only let the developer specify major parameters.  Notice how FactoryDerivative.sobel() does not allow the user to specify the border type, that's because it assumes it knows the best type and might as well spare the developer from that level of detail.


= No Generics =
= No Generics =
Line 122: Line 120:


<syntaxhighlight lang="java">
<syntaxhighlight lang="java">
public static void nogenerics( ImageBase input )
public static void nogenerics( ImageGray input )
{
{
Class inputType = input.getClass();
Class inputType = input.getClass();
Class derivType = GImageDerivativeOps.getDerivativeType(inputType);
Class derivType = GImageDerivativeOps.getDerivativeType(inputType);


ImageBase blurred = GeneralizedImageOps.createImage(inputType,input.width, input.height);
ImageGray blurred = (ImageGray)input.createSameShape();
ImageBase derivX = GeneralizedImageOps.createImage(derivType,input.width, input.height);
ImageGray derivX = GeneralizedImageOps.createSingleBand(derivType, input.width, input.height);
ImageBase derivY = GeneralizedImageOps.createImage(derivType,input.width, input.height);
ImageGray derivY = GeneralizedImageOps.createSingleBand(derivType, input.width, input.height);


// Gaussian blur: Convolve a Gaussian kernel with a width of 5 pixels
// Gaussian blur: Convolve a Gaussian kernel
GBlurImageOps.gaussian(input, blurred, -1, 2, null);
GBlurImageOps.gaussian(input, blurred, -1, blurRadius, null);


// Calculate image's derivative
// Calculate image's derivative
GImageDerivativeOps.sobel(blurred, derivX, derivY, BorderType.EXTENDED);
GImageDerivativeOps.gradient(DerivativeType.SOBEL,blurred, derivX, derivY, BorderType.EXTENDED);


// display the results
// display the results
BufferedImage outputImage = VisualizeImageData.colorizeSign(derivX,null,-1);
BufferedImage outputImage = VisualizeImageData.colorizeGradient(derivX, derivY,-1, null);
ShowImages.showWindow(outputImage,"Generalized "+inputType.getSimpleName());
panel.addImage(outputImage,"Generalized "+inputType.getSimpleName());
}
}
</syntaxhighlight>
</syntaxhighlight>

Revision as of 07:34, 15 March 2019

This tutorial introduces basic BoofCV programming concepts through three examples. In BoofCV there are often three ways to invoke a function; 1) procedural, 2) generalized, and 3) abstracted.

Procedural functions are contained in children of the boofcv.alg package, where 'alg' stands for algorithms. These procedural functions are low level implementations which allow greater customization, tighter memory control, and have strong typing. In the same package are generalized versions of procedural functions that provide weaker type checking, but allow generic images to be passed in. Classes which contain generalized functions start with the letter 'G'. For example, BlurImageOps contains a set of functions for each image type and operation pair it supports and GBlurImageOps has a single function for each operation it supports.

Inside the children of 'boofcv.abst' package are abstracted filters. These classes provide an Object Oriented Programming (OOP) interface and handle much of the memory management, but are less flexible. Some times the ability to configure low level parameters has been removed from the algorithms that they are wrappers around to make development more manageable.

Example File: ExampleImageFilter

Tutorial Concepts:

  1. Invoke procedural functions.
  2. Invoke generalized functions using Java generics.
  3. Create and invoke abstracted filters.
  4. Use a Factory.
  5. Display images.

Relevant Examples:

  1. Image Blur
  2. Image Derivative

Procedural

Here an GrayU8 image is passed in, which is then blurred, has its derivative computed and displayed in a window. In BoofCV, unlike many other vision libraries, to provide strong type checking at compile time, images with different primitive data types are a different type. The name 'U8' indicates that its elements are unsigned 8-bit integers and that it has only a single band. Image derivative require a different data type because the domain of each element includes negative numbers and has a larger domain, which is why they are stored in a GrayS16. GrayS16 are single band images with signed 16-bit integer pixels.

	public static void procedural( GrayU8 input )
	{
		GrayU8 blurred = new GrayU8(input.width,input.height);
		GrayS16 derivX = new GrayS16(input.width,input.height);
		GrayS16 derivY = new GrayS16(input.width,input.height);

		// Gaussian blur: Convolve a Gaussian kernel
		BlurImageOps.gaussian(input,blurred,-1,blurRadius,null);

		// Calculate image's derivative
		GradientSobel.process(blurred, derivX, derivY, FactoryImageBorderAlgs.extend(input));

		// display the results
		BufferedImage outputImage = VisualizeImageData.colorizeGradient(derivX, derivY, -1, null);
		panel.addImage(outputImage,"Procedural Fixed Type");
	}

The two integer parameters in BlurImageOps.gaussian(input,blurred,-1,2,null) specify the blur's sigma and radius. A negative sigma indicates that the sigma should be chosen based upon the kernel's radius. The converse is also true, a negative radius means the radius should be selected based upon the sigma's value. When applying Gaussian blur, additional storage is required. Because null was passed in for the last parameter, it will internally declare that memory. Declaring temporary memory each time a function is invoked is wasteful, but easier to program.

When convolving a kernel across an image its not always clear how image borders should be handled. At the image border convolution kernels will be partially outside of the image. In this example, the gradient is computed using a Sobel kernel, and BorderType.EXTENDED tells it to handle the border by extending border pixels to outside the image.

Finally the output is computed by rendering the image derivative into a BufferedImage. BufferedImages come from Java Swing and is Java's standard way for storing image data. They are not used internally in BoofCV due to their complexity and poor performance. Once rendered, it is displayed in an image using the convenient showWindow() function.

Generalized

The following example is very similar to the first, but it now takes in a generalized image type using Java generics. Type checking is weaker, but different image types can be passed in using the exact same code.

	public static <T extends ImageGray<T>, D extends ImageGray<D>>
	void generalized( T input )
	{
		Class<T> inputType = (Class<T>)input.getClass();
		Class<D> derivType = GImageDerivativeOps.getDerivativeType(inputType);

		T blurred = (T)input.createSameShape();
		D derivX = GeneralizedImageOps.createSingleBand(derivType, input.width, input.height);
		D derivY = GeneralizedImageOps.createSingleBand(derivType, input.width, input.height);

		// Gaussian blur: Convolve a Gaussian kernel
		GBlurImageOps.gaussian(input, blurred, -1, blurRadius, null);

		// Calculate image's derivative
		GImageDerivativeOps.gradient(DerivativeType.SOBEL,blurred, derivX, derivY, BorderType.EXTENDED);

		// display the results
		BufferedImage outputImage = VisualizeImageData.colorizeGradient(derivX, derivY,-1, null);
		panel.addImage(outputImage,"Generalized "+inputType.getSimpleName());
	}

Since the type is not known at compile type, images are created using GeneralizedImageOps.createImage(). Java generics allow consistency checks to make sure images which should be the same type are the same type. Notice that blurring and image derivatives are computed using classes that begin with the letter 'G', that signifies they are generalized classes.

Abstracted

Instead of invoking functions, filter classes are created here. Filters are even more flexible and allow greater abstraction.

	public static <T extends ImageGray<T>, D extends ImageGray<D>>
	void filter( T input )
	{
		Class<T> inputType = (Class<T>)input.getClass();
		Class<D> derivType = GImageDerivativeOps.getDerivativeType(inputType);

		T blurred = (T)input.createSameShape();
		D derivX = GeneralizedImageOps.createSingleBand(derivType, input.width, input.height);
		D derivY = GeneralizedImageOps.createSingleBand(derivType, input.width, input.height);

		// declare image filters
		BlurFilter<T> filterBlur = FactoryBlurFilter.gaussian(ImageType.single(inputType), -1, blurRadius);
		ImageGradient<T,D> gradient = FactoryDerivative.sobel(inputType, derivType);

		// process the image
		filterBlur.process(input,blurred);
		gradient.process(blurred,derivX,derivY);

		// display the results
		BufferedImage outputImage = VisualizeImageData.colorizeGradient(derivX, derivY, -1, null);
		panel.addImage(outputImage,"Filter "+inputType.getSimpleName());
	}

Notice how Factories are used to create filters. Factories provide an easy to use interface for constructing filters and other algorithms. They also provide a single location to look for similar algorithms which implement the same interface. Often they will simplify development by handling all the little details and only let the developer specify major parameters. Notice how FactoryDerivative.sobel() does not allow the user to specify the border type, that's because it assumes it knows the best type and might as well spare the developer from that level of detail.

No Generics

For sake of completeness, it is demonstrated that Java generics are optional when writing generalized code. No type checking is done here, but the code is less verbose.

	public static void nogenerics( ImageGray input )
	{
		Class inputType = input.getClass();
		Class derivType = GImageDerivativeOps.getDerivativeType(inputType);

		ImageGray blurred = (ImageGray)input.createSameShape();
		ImageGray derivX = GeneralizedImageOps.createSingleBand(derivType, input.width, input.height);
		ImageGray derivY = GeneralizedImageOps.createSingleBand(derivType, input.width, input.height);

		// Gaussian blur: Convolve a Gaussian kernel
		GBlurImageOps.gaussian(input, blurred, -1, blurRadius, null);

		// Calculate image's derivative
		GImageDerivativeOps.gradient(DerivativeType.SOBEL,blurred, derivX, derivY, BorderType.EXTENDED);

		// display the results
		BufferedImage outputImage = VisualizeImageData.colorizeGradient(derivX, derivY,-1, null);
		panel.addImage(outputImage,"Generalized "+inputType.getSimpleName());
	}