Tutorial Images

Images in BoofCV

Its not surprising that images play a key role inside of BoofCV, since its a computer vision library. Images are 2D arrays which store pixel intensity values, where a single pixel represents the output of a optical sensor. In BoofCV to help catch more errors at compile time and to ensure more meaningful error messages it provides several different types of images. The more standard approach (such as is done in Java2D or OpenCV) is to provide a single data type which can represent many different image types.

There is one image type for each primitive data structure, including signed and unsigned data types. For example, ImageFloat32 and ImageUInt8 represent single band images containing 32-bit floats and unsigned 8-bit pixels, respectively. At the time of this writing, BoofCV primarily supports only single band or gray scale images with some support for color image processing. The reason for this is because most computer vision routines are only designed for gray scale images.

Naming Scheme:

Single Band Floating Point Images

ImageFloat*

* indicates the number of bits. Only 32 bits and 64 bits are supported.

Single Band Integer Images

Image*Int*

The first * can be S or U for signed or unsigned.

The second * is the number of bits and can be 8,16,32, or 64.

A different naming scheme is used for multi-band images but is not yet discussed here since it has only minimal support in BoofCV and is subject to change.

Why not use BufferedImage?

Images in BoofCV are designed for high speed image processing. Extensive benchmarking has shown that in Java (counter to what many people believe) using accessors (e.g. get() or set()) will slow down your low level image processing code. It is still true that in high level applications accessors make no performance difference. For this reason, data structures in images are publicly accessible.

It is much faster and easier to work with images in BoofCV than the Java2D data structure BufferedImage. BufferedImages are complex and accessing pixels through their generalized interfaces is very slow. BoofCV provides several routines for quickly converting BufferedImage into BoofCV images. Using BoofCV routines for conversion is very fast because they access low level rasters inside of BufferedImage and gracefully handle situations where access is blocked to the rasters (e.g. Applets).

Image Types

Class inheritance diagram for high level image types

BoofCV supports both color and single band images and all image types extend the abstract ImageBase class. ImageBase proves basic information on the size, shape, and internal structure of the class. Immediate children of ImageBase is ImageSingleBand and MultiSpectral, which provide support for gray scale and color image processing, respectively.

ImageSingleBand contain only one intensity value per pixel and can also be referred to as a gray scale image. MultiSpectral contains each color band as its own gray scale image. Any operation which can process a ImageSingleBand can process one of the bands inside of a MultiSpectral image, allowing for fast implementation of some color image processing and computer vision algorithms. In the future interleaved color images will be supported.

Single Band Images

Class inheritance diagram for single band images

Single band-images are gray scale images which represent an image using a single color or band. They are easier to work with the multi-band data and more commonly used in computer vision because they are faster to work with. The down side to working with gray scale images is that to create one often requires throwing away valuable color information.

Data Structure

All images in BoofCV extend ImageBase and have the following data structure:

public int startIndex;
public int stride;
public int width;
public int height;
public DATA_TYPE data;

where DATA_TYPE depends upon the type of primitive data stored inside the image. 'startIndex' is the first index in 'data' that represents the first pixel in the image. 'stride' is how many pixels between rows in the image. 'width' and 'height' are the number of columns and rows in the image. 'data' is where pixel intensity values are stored. Pixel information is stored in a row-major format.

Pixel Access

The easiest way to access a pixel is using the set() and get() functions. However pixels can also be accessed directly.

ImageFloat32 image = new ImageFloat32(60,60);

int x = 10;
int y = 20;

// using accessors
image.set(x,y,45);
float value1 = image.get(x,y);

// direct access
float value2 = image.data[ image.startIndex + y*image.stride + x];

System.out.println(value1+" "+value2);

The advantage to using accessors is that they are easier to use and safe. Accessors perform bounds checking and for unsigned integer images they do the necessary bitwise operations to make sure the returned data is unsigned. Example of accessing pixel information in an unsigned integer image below:

ImageUInt8 image = new ImageUInt8(60,60);
image.set(x,y,45);
int value = image.data[ image.startIndex + y*image.stride + x] & 0xFF;

Multi Spectral Images

MultiSpectral images are color images where each color band is stored as a separate gray scale image. Storing color images in this format is less common than the typical interleaved format, but allow color images to be easily processed using highly optimized gray scale code that is common throughout BoofCV. The main disadvantage is that for some algorithm types an interleaved image will be faster to process when color information can not be processes in each band independently.

Working with MultiSpectral images is easy. To access the intensity of a pixel in different colors, simply access the image for that band then use the gray scale accessors functions. Below a code sniplet shows how to create a multi spectral image and set the pixel at (5,5) to have a value of (255,50,50), which would appear to be red in an RGB image.

MultiSpectral<ImageUInt8> image = new MultiSpectral<ImageUInt8>(ImageUInt8.class,100,200,3);
image.getBand(0).set(5,5,255);
image.getBand(1).set(5,5,50);
image.getBand(2).set(5,5,50);

Sub-Images

A sub-image is an image which is a rectangular region inside a larger image. They are useful when only a part of an image needs to be processed, e.g. region of interest, or when only part of the image should be modified. Creating sub-image is easy, just call the subimage() function, which is part of all images. Any changes you make to a sub-image will affect the larger image too.

	public static void main( String args[] ) {
		ImageFloat32 image = new ImageFloat32(100,100);
		ImageFloat32 sub = image.subimage(10,15,30,30);

		sub.set(0,0,100);

		System.out.println(sub.get(0,0));
		System.out.println(image.get(10,15));
	}

This outputs "100" in both print statements because the call to set() affects the sub-image and the original image.

Generics

Generics can be used to improve the readability and abstractness of code. Images in BoofCV use Java generics to provide better type checking in abstracted code.

Type specific code:

	public static ImageUInt8 typeSpecific( ImageUInt8 image ) {
		ImageUInt8 output = new ImageUInt8(image.width,image.height);
		
		BlurImageOps.gaussian(image,output,-1,2,null);
		
		return output;
	}

Abstracted code using generics:

	public static <T extends ImageBase> T generic( T image ) {
		T output = (T)image._createNew(image.width,image.height);
		
		GBlurImageOps.gaussian(image, output, -1, 2, null);
		
		return output;
	}

One disadvantage of working with generics is that not all image type errors can be caught at runtime. In the second example above it is possible to pass in an image not supported by GBlurImageOps, which will cause a RuntimeException to be thrown. At the same time it is possible to use generics to only allow a specific type of image.