Difference between revisions of "Tutorial Images"

From BoofCV
Jump to navigationJump to search
m
 
(11 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Images in BoofCV =
Its not surprising that images play a key role inside of BoofCV, since its a computer vision library.  Images are 2D arrays which store pixel intensity values, where a single pixel represents the output of a optical sensor.  In BoofCV, to help catch more errors at compile time and to ensure more meaningful error messages, several different types of images are provided.  The more standard approach (such as is done in Java2D or OpenCV) is to provide a single data type which can represent many different image types.


Because BoofCV is a computer vision library, its primary data structures imagesImages are 2D arrays which store the intensity of pixel values, where a single pixel represents the output of a optical sensorIn BoofCV to help catch more errors at compile time and to ensure more meaningful error messages it provides several different types of imagesThe more standard approach (such as is done in Javs2D or OpenCV) is to provide a single data type which can represent many different image types.
There is one image type for each primitive data element, including signed and unsigned data typesFor example, GrayF32 and GrayU8 represent single band images containing 32-bit floats and unsigned 8-bit integer pixels, respectivelyTwo color image formats are supported, Planar and Interleaved in different parts of the codeWhich one is supported depends on which one is the most advantageous for the algorithm.


There is one image type for each primitive data structure, including signed and unsigned data types. For example, ImageFloat32 and ImageUInt8 represent single band images containing 32-bit floats and unsigned 8-bit pixels, respectively.  At the time of this writing, BoofCV primarily supports only single band or gray scale images.  The reason for this is because most computer vision routines are only designed for gray scale images, mostly for speed reasons.
The type of primitive data structure contained in a class is specified with a suffix.  Not all data types are supported due to language limitations found in Java, e.g. no unsigned 64-bit integer.
; U8
: Unsigned 8-bit integer
; S8
: Signed 8-bit integer
; U16
: Unsigned 16-bit integer
; S16
: Signed 16-bit integer
; S32
: Signed 32-bit integer
; S64
: Signed 64-bit integer
; F32
: Floating point 32-bit
; F64
: Floating point 64-bit


Naming Scheme:
; Single Band Floating Point Images
: ImageFloat*
: * indicates the number of bits. Only 32 bits and 64 bits are supported.
; Single Band Integer Images
: Image*Int*
: The first * can be S or U for signed or unsigned.
: The second * is the number of bits and can be 8,16,32, or 64.


A different naming scheme is used for multi-band images but is not yet discussed here since it has only minimal support in BoofCV and is subject to change.
= Why not use BufferedImage? =
Images in BoofCV are designed for high speed image processing with simple data structures for easier development.  BufferedImages are highly abstracted and designed to encapsulate just about any image format.  Manipulating images using BufferedImage's getRGB() and setRGB() functions is painfully slow.  In-fact the only way to get any speed out of BufferedImages is to use the forbidden low level interfaces!  Compared to BoofCV where its images are highly type specific, with absolutely minimal abstraction and strong typing.  In BoofCV, raw pixel data can even manipulated without accessors functions, for added speed in low level functions easily.
 
BoofCV provides several routines for quickly converting BufferedImage into BoofCV images.  Using BoofCV routines for converting to/from BufferedImages are very fast because they access low level rasters inside of BufferedImage and gracefully handle situations where access is blocked to the rasters (e.g. Applets).
 
= Image Families =
 
{|
|-
! Class Inheritance Diagram for High Level Image Types
|-
| [[File:Image_type_class_diagram.png]]
|}


= Why not use BufferedImage? =
All image types extend the abstract ImageBase class.  ImageBase proves basic information on the size, shape, and internal structure of the class.  Immediate children of ImageBase are ImageGray, Interleaved, and Planar.  ImageGray contain only one intensity value per pixel and can also be referred to as a gray scale image.  Interleaved stores color information in adjacent elements for the same pixel. Planar contains each color band as its own gray scale image.  Once advantage to Planar images is that the individual bands can be treated as ImageGray, which is exactly what they are.
Images in BoofCV are designed for high speed image processingExtensive benchmarking has shown that in Java (counter to what many people believe) using accessors (e.g. get() or set()) will slow down your low level image processing codeIt is still true that in high level applications accessors make no performance difference.  For this reason, data structures in images are publicly accessible.


It is much faster and easier to work with images in BoofCV than the Java2D data structure BufferedImage.  BufferedImages are complex and accessing pixels through their generalized interfaces is very slow.  BoofCV provides several routines for quickly converting BufferedImage into BoofCV images.  Using BoofCV routines for conversion is very fast because they access low level rasters inside of BufferedImage and gracefully handle situations where access is blocked to the rasters (e.g. Applets).
= Image Gray =


= Single Band Images =
{|
|-
! Class Inheritance Diagram for ImageGray
|-
| [[File:Image_type_single_class_diagram.png]]
|}


Single band-images are gray scale images which represent an image using a single color or band.  They are easier to work with the multi-band data and more commonly used in computer vision because they are faster to work with.  The down side to working with gray scale images is that to create one often requires throwing away valuable color information.
Gray images are single-band images (a.k.a. gray scale images) which represent an image using a single color or band.  They are easier to work with the multi-band data and more commonly used in computer vision because they are faster to work with and much of the visual information is contained in a single band.   


== Data Structure ==
== Internal Data Structure ==


All images in BoofCV extend ImageBase and have the following data structure:
All images in BoofCV extend ImageBase and has the following structure:
<pre>
<pre>
public int startIndex;
public int startIndex;
Line 41: Line 65:
The easiest way to access a pixel is using the set() and get() functions.  However pixels can also be accessed directly.
The easiest way to access a pixel is using the set() and get() functions.  However pixels can also be accessed directly.
<pre>
<pre>
ImageFloat32 image = new ImageFloat32(60,60);
GrayF32 image = new GrayF32(60,60);


int x = 10;
int x = 10;
Line 59: Line 83:


<pre>
<pre>
ImageUInt8image = new ImageUInt8image (60,60);
GrayU8 image = new GrayU8(60,60);
image.set(x,y,45);
image.set(x,y,45);
int value = image.data[ image.startIndex + y*image.stride + x] & 0xFF;
int value = image.data[ image.startIndex + y*image.stride + x] & 0xFF;
</pre>
= Interleaved Images =
{|
|-
! Class Inheritance Diagram for Interleaved Images
|-
| [[File:Image_type_interleaved_diagram.png]]
|}
= Planar Images =
Planar images are color images where each color band is stored as a separate gray scale image. Storing color images in this format is less common than the typical interleaved format, but allows color images to be easily processed using highly optimized gray scale code that is common throughout BoofCV.
Working with Planar images is easy.  To access the intensity of a pixel in different colors, simply access the image for that band then use the gray scale accessors functions.  Below a code sniplet shows how to create a multi spectral image and set the pixel at (5,5) to have a value of (255,50,50), which would appear to be red in an RGB image.
<pre>
Planar<GrayU8> image = new Planar<GrayU8>(GrayU8.class,100,200,3);
image.getBand(0).set(5,5,255);
image.getBand(1).set(5,5,50);
image.getBand(2).set(5,5,50);
</pre>
</pre>


Line 70: Line 118:
<pre>
<pre>
public static void main( String args[] ) {
public static void main( String args[] ) {
ImageFloat32 image = new ImageFloat32(100,100);
GrayF32 image = new GrayF32(100,100);
ImageFloat32 sub = image.subimage(10,15,30,30);
GrayF32 sub = image.subimage(10,15,30,30);


sub.set(0,0,100);
sub.set(0,0,100);
Line 87: Line 135:
Type specific code:
Type specific code:
<pre>
<pre>
public static ImageUInt8 typeSpecific( ImageUInt8 image ) {
public static GrayU8 typeSpecific( GrayU8 image ) {
ImageUInt8 output = new ImageUInt8(image.width,image.height);
GrayU8 output = new GrayU8(image.width,image.height);
BlurImageOps.gaussian(image,output,-1,2,null);
BlurImageOps.gaussian(image,output,-1,2,null);

Latest revision as of 08:15, 29 March 2016

Its not surprising that images play a key role inside of BoofCV, since its a computer vision library. Images are 2D arrays which store pixel intensity values, where a single pixel represents the output of a optical sensor. In BoofCV, to help catch more errors at compile time and to ensure more meaningful error messages, several different types of images are provided. The more standard approach (such as is done in Java2D or OpenCV) is to provide a single data type which can represent many different image types.

There is one image type for each primitive data element, including signed and unsigned data types. For example, GrayF32 and GrayU8 represent single band images containing 32-bit floats and unsigned 8-bit integer pixels, respectively. Two color image formats are supported, Planar and Interleaved in different parts of the code. Which one is supported depends on which one is the most advantageous for the algorithm.

The type of primitive data structure contained in a class is specified with a suffix. Not all data types are supported due to language limitations found in Java, e.g. no unsigned 64-bit integer.

U8
Unsigned 8-bit integer
S8
Signed 8-bit integer
U16
Unsigned 16-bit integer
S16
Signed 16-bit integer
S32
Signed 32-bit integer
S64
Signed 64-bit integer
F32
Floating point 32-bit
F64
Floating point 64-bit


Why not use BufferedImage?

Images in BoofCV are designed for high speed image processing with simple data structures for easier development. BufferedImages are highly abstracted and designed to encapsulate just about any image format. Manipulating images using BufferedImage's getRGB() and setRGB() functions is painfully slow. In-fact the only way to get any speed out of BufferedImages is to use the forbidden low level interfaces! Compared to BoofCV where its images are highly type specific, with absolutely minimal abstraction and strong typing. In BoofCV, raw pixel data can even manipulated without accessors functions, for added speed in low level functions easily.

BoofCV provides several routines for quickly converting BufferedImage into BoofCV images. Using BoofCV routines for converting to/from BufferedImages are very fast because they access low level rasters inside of BufferedImage and gracefully handle situations where access is blocked to the rasters (e.g. Applets).

Image Families

Class Inheritance Diagram for High Level Image Types
Image type class diagram.png

All image types extend the abstract ImageBase class. ImageBase proves basic information on the size, shape, and internal structure of the class. Immediate children of ImageBase are ImageGray, Interleaved, and Planar. ImageGray contain only one intensity value per pixel and can also be referred to as a gray scale image. Interleaved stores color information in adjacent elements for the same pixel. Planar contains each color band as its own gray scale image. Once advantage to Planar images is that the individual bands can be treated as ImageGray, which is exactly what they are.

Image Gray

Class Inheritance Diagram for ImageGray
Image type single class diagram.png

Gray images are single-band images (a.k.a. gray scale images) which represent an image using a single color or band. They are easier to work with the multi-band data and more commonly used in computer vision because they are faster to work with and much of the visual information is contained in a single band.

Internal Data Structure

All images in BoofCV extend ImageBase and has the following structure:

public int startIndex;
public int stride;
public int width;
public int height;
public DATA_TYPE data;

where DATA_TYPE depends upon the type of primitive data stored inside the image. 'startIndex' is the first index in 'data' that represents the first pixel in the image. 'stride' is how many pixels between rows in the image. 'width' and 'height' are the number of columns and rows in the image. 'data' is where pixel intensity values are stored. Pixel information is stored in a row-major format.

Pixel Access

The easiest way to access a pixel is using the set() and get() functions. However pixels can also be accessed directly.

GrayF32 image = new GrayF32(60,60);

int x = 10;
int y = 20;

// using accessors
image.set(x,y,45);
float value1 = image.get(x,y);

// direct access
float value2 = image.data[ image.startIndex + y*image.stride + x];

System.out.println(value1+" "+value2);

The advantage to using accessors is that they are easier to use and safe. Accessors perform bounds checking and for unsigned integer images they do the necessary bitwise operations to make sure the returned data is unsigned. Example of accessing pixel information in an unsigned integer image below:

GrayU8 image = new GrayU8(60,60);
image.set(x,y,45);
int value = image.data[ image.startIndex + y*image.stride + x] & 0xFF;

Interleaved Images

Class Inheritance Diagram for Interleaved Images
Image type interleaved diagram.png


Planar Images

Planar images are color images where each color band is stored as a separate gray scale image. Storing color images in this format is less common than the typical interleaved format, but allows color images to be easily processed using highly optimized gray scale code that is common throughout BoofCV.

Working with Planar images is easy. To access the intensity of a pixel in different colors, simply access the image for that band then use the gray scale accessors functions. Below a code sniplet shows how to create a multi spectral image and set the pixel at (5,5) to have a value of (255,50,50), which would appear to be red in an RGB image.

Planar<GrayU8> image = new Planar<GrayU8>(GrayU8.class,100,200,3);
image.getBand(0).set(5,5,255);
image.getBand(1).set(5,5,50);
image.getBand(2).set(5,5,50);

Sub-Images

A sub-image is an image which is a rectangular region inside a larger image. They are useful when only a part of an image needs to be processed, e.g. region of interest, or when only part of the image should be modified. Creating sub-image is easy, just call the subimage() function, which is part of all images. Any changes you make to a sub-image will affect the larger image too.

	public static void main( String args[] ) {
		GrayF32 image = new GrayF32(100,100);
		GrayF32 sub = image.subimage(10,15,30,30);

		sub.set(0,0,100);

		System.out.println(sub.get(0,0));
		System.out.println(image.get(10,15));
	}

This outputs "100" in both print statements because the call to set() affects the sub-image and the original image.

Generics

Generics can be used to improve the readability and abstractness of code. Images in BoofCV use Java generics to provide better type checking in abstracted code.

Type specific code:

	public static GrayU8 typeSpecific( GrayU8 image ) {
		GrayU8 output = new GrayU8(image.width,image.height);
		
		BlurImageOps.gaussian(image,output,-1,2,null);
		
		return output;
	}

Abstracted code using generics:

	public static <T extends ImageBase> T generic( T image ) {
		T output = (T)image._createNew(image.width,image.height);
		
		GBlurImageOps.gaussian(image, output, -1, 2, null);
		
		return output;
	}

One disadvantage of working with generics is that not all image type errors can be caught at runtime. In the second example above it is possible to pass in an image not supported by GBlurImageOps, which will cause a RuntimeException to be thrown. At the same time it is possible to use generics to only allow a specific type of image.