Tutorial Images

Its not surprising that images play a key role inside of BoofCV, since its a computer vision library. Images are 2D arrays which store pixel intensity values, where a single pixel represents the output of a optical sensor. In BoofCV, to help catch more errors at compile time and to ensure more meaningful error messages, several different types of images are provided. The more standard approach (such as is done in Java2D or OpenCV) is to provide a single data type which can represent many different image types.

There is one image type for each primitive data element, including signed and unsigned data types. For example, GrayF32 and GrayU8 represent single band images containing 32-bit floats and unsigned 8-bit integer pixels, respectively. Two color image formats are supported, Planar and Interleaved in different parts of the code. Which one is supported depends on which one is the most advantageous for the algorithm.

The type of primitive data structure contained in a class is specified with a suffix. Not all data types are supported due to language limitations found in Java, e.g. no unsigned 64-bit integer.

U8

Unsigned 8-bit integer

S8

Signed 8-bit integer

U16

Unsigned 16-bit integer

S16

Signed 16-bit integer

S32

Signed 32-bit integer

S64

Signed 64-bit integer

F32

Floating point 32-bit

F64

Floating point 64-bit

Why not use BufferedImage?

Images in BoofCV are designed for high speed image processing with simple data structures for easier development. BufferedImages are highly abstracted and designed to encapsulate just about any image format. Manipulating images using BufferedImage's getRGB() and setRGB() functions is painfully slow. In-fact the only way to get any speed out of BufferedImages is to use the forbidden low level interfaces! Compared to BoofCV where its images are highly type specific, with absolutely minimal abstraction and strong typing. In BoofCV, raw pixel data can even manipulated without accessors functions, for added speed in low level functions easily.

BoofCV provides several routines for quickly converting BufferedImage into BoofCV images. Using BoofCV routines for converting to/from BufferedImages are very fast because they access low level rasters inside of BufferedImage and gracefully handle situations where access is blocked to the rasters (e.g. Applets).

Image Families

Class Inheritance Diagram for High Level Image Types

All image types extend the abstract ImageBase class. ImageBase proves basic information on the size, shape, and internal structure of the class. Immediate children of ImageBase are ImageGray, Interleaved, and Planar. ImageGray contain only one intensity value per pixel and can also be referred to as a gray scale image. Interleaved stores color information in adjacent elements for the same pixel. Planar contains each color band as its own gray scale image. Once advantage to Planar images is that the individual bands can be treated as ImageGray, which is exactly what they are.

Image Gray

Class Inheritance Diagram for ImageGray

Gray images are single-band images (a.k.a. gray scale images) which represent an image using a single color or band. They are easier to work with the multi-band data and more commonly used in computer vision because they are faster to work with and much of the visual information is contained in a single band.

Internal Data Structure

All images in BoofCV extend ImageBase and has the following structure:

public int startIndex;
public int stride;
public int width;
public int height;
public DATA_TYPE data;

where DATA_TYPE depends upon the type of primitive data stored inside the image. 'startIndex' is the first index in 'data' that represents the first pixel in the image. 'stride' is how many pixels between rows in the image. 'width' and 'height' are the number of columns and rows in the image. 'data' is where pixel intensity values are stored. Pixel information is stored in a row-major format.

Pixel Access

The easiest way to access a pixel is using the set() and get() functions. However pixels can also be accessed directly.

ImageFloat32 image = new ImageFloat32(60,60);

int x = 10;
int y = 20;

// using accessors
image.set(x,y,45);
float value1 = image.get(x,y);

// direct access
float value2 = image.data[ image.startIndex + y*image.stride + x];

System.out.println(value1+" "+value2);

The advantage to using accessors is that they are easier to use and safe. Accessors perform bounds checking and for unsigned integer images they do the necessary bitwise operations to make sure the returned data is unsigned. Example of accessing pixel information in an unsigned integer image below:

GrayU8 image = new GrayU8(60,60);
image.set(x,y,45);
int value = image.data[ image.startIndex + y*image.stride + x] & 0xFF;

Interleaved Images

Class Inheritance Diagram for Interleaved Images

Planar Images

Planar images are color images where each color band is stored as a separate gray scale image. Storing color images in this format is less common than the typical interleaved format, but allows color images to be easily processed using highly optimized gray scale code that is common throughout BoofCV.

Working with Planar images is easy. To access the intensity of a pixel in different colors, simply access the image for that band then use the gray scale accessors functions. Below a code sniplet shows how to create a multi spectral image and set the pixel at (5,5) to have a value of (255,50,50), which would appear to be red in an RGB image.

Planar<GrayU8> image = new Planar<GrayU8>(GrayU8.class,100,200,3);
image.getBand(0).set(5,5,255);
image.getBand(1).set(5,5,50);
image.getBand(2).set(5,5,50);

Sub-Images

A sub-image is an image which is a rectangular region inside a larger image. They are useful when only a part of an image needs to be processed, e.g. region of interest, or when only part of the image should be modified. Creating sub-image is easy, just call the subimage() function, which is part of all images. Any changes you make to a sub-image will affect the larger image too.

	public static void main( String args[] ) {
		GrayF32 image = new GrayF32(100,100);
		GrayF32 sub = image.subimage(10,15,30,30);

		sub.set(0,0,100);

		System.out.println(sub.get(0,0));
		System.out.println(image.get(10,15));
	}

This outputs "100" in both print statements because the call to set() affects the sub-image and the original image.

Generics

Generics can be used to improve the readability and abstractness of code. Images in BoofCV use Java generics to provide better type checking in abstracted code.

Type specific code:

	public static GrayU8 typeSpecific( GrayU8 image ) {
		GrayU8 output = new GrayU8(image.width,image.height);
		
		BlurImageOps.gaussian(image,output,-1,2,null);
		
		return output;
	}

Abstracted code using generics:

	public static <T extends ImageBase> T generic( T image ) {
		T output = (T)image._createNew(image.width,image.height);
		
		GBlurImageOps.gaussian(image, output, -1, 2, null);
		
		return output;
	}

One disadvantage of working with generics is that not all image type errors can be caught at runtime. In the second example above it is possible to pass in an image not supported by GBlurImageOps, which will cause a RuntimeException to be thrown. At the same time it is possible to use generics to only allow a specific type of image.