Difference between revisions of "Tutorial Images"

From BoofCV
Jump to navigationJump to search
(Cleaned up writing a bit)
m
 
(4 intermediate revisions by the same user not shown)
Line 1: Line 1:
Its not surprising that images play a key role inside of BoofCV, since its a computer vision library.  Images are 2D arrays which store pixel intensity values, where a single pixel represents the output of a optical sensor.  In BoofCV to help catch more errors at compile time and to ensure more meaningful error messages it provides several different types of images.  The more standard approach (such as is done in Java2D or OpenCV) is to provide a single data type which can represent many different image types.
Its not surprising that images play a key role inside of BoofCV, since its a computer vision library.  Images are 2D arrays which store pixel intensity values, where a single pixel represents the output of a optical sensor.  In BoofCV, to help catch more errors at compile time and to ensure more meaningful error messages, several different types of images are provided.  The more standard approach (such as is done in Java2D or OpenCV) is to provide a single data type which can represent many different image types.


There is one image type for each primitive data structure, including signed and unsigned data types.  For example, ImageFloat32 and ImageUInt8 represent single band images containing 32-bit floats and unsigned 8-bit pixels, respectively.  At the time of this writing, BoofCV primarily supports only single band or gray scale images with some support for color image processingThe reason for this is because most computer vision routines are only designed for gray scale images.
There is one image type for each primitive data element, including signed and unsigned data types.  For example, GrayF32 and GrayU8 represent single band images containing 32-bit floats and unsigned 8-bit integer pixels, respectively.  Two color image formats are supported, Planar and Interleaved in different parts of the codeWhich one is supported depends on which one is the most advantageous for the algorithm.


Naming Scheme:
The type of primitive data structure contained in a class is specified with a suffix.  Not all data types are supported due to language limitations found in Java, e.g. no unsigned 64-bit integer.
; Single Band Floating Point Images
; U8
: ImageFloat*
: Unsigned 8-bit integer
: * indicates the number of bits. Only 32 bits and 64 bits are supported.
; S8
; Single Band Integer Images
: Signed 8-bit integer
: Image*Int*
; U16
: The first * can be S or U for signed or unsigned.
: Unsigned 16-bit integer
: The second * is the number of bits and can be 8,16,32, or 64.
; S16
: Signed 16-bit integer
; S32
: Signed 32-bit integer
; S64
: Signed 64-bit integer
; F32
: Floating point 32-bit
; F64
: Floating point 64-bit


A different naming scheme is used for multi-band images but is not yet discussed here since it has only minimal support in BoofCV and is subject to change.


= Why not use BufferedImage? =
= Why not use BufferedImage? =
Line 19: Line 27:
BoofCV provides several routines for quickly converting BufferedImage into BoofCV images.  Using BoofCV routines for converting to/from BufferedImages are very fast because they access low level rasters inside of BufferedImage and gracefully handle situations where access is blocked to the rasters (e.g. Applets).
BoofCV provides several routines for quickly converting BufferedImage into BoofCV images.  Using BoofCV routines for converting to/from BufferedImages are very fast because they access low level rasters inside of BufferedImage and gracefully handle situations where access is blocked to the rasters (e.g. Applets).


= Image Types =
= Image Families =


{|
{|
|-
|-
! Class inheritance diagram for high level image types
! Class Inheritance Diagram for High Level Image Types
|-
|-
| [[File:Image_type_class_diagram.png]]
| [[File:Image_type_class_diagram.png]]
|}
|}


BoofCV supports both color and single band images.  All image types extend the abstract ImageBase class.  ImageBase proves basic information on the size, shape, and internal structure of the class.  Immediate children of ImageBase are ImageSingleBand and MultiSpectral, which provide support for gray scale and color image processing, respectively.
All image types extend the abstract ImageBase class.  ImageBase proves basic information on the size, shape, and internal structure of the class.  Immediate children of ImageBase are ImageGray, Interleaved, and Planar.  ImageGray contain only one intensity value per pixel and can also be referred to as a gray scale image.  Interleaved stores color information in adjacent elements for the same pixel.  Planar contains each color band as its own gray scale image.  Once advantage to Planar images is that the individual bands can be treated as ImageGray, which is exactly what they are.


ImageSingleBand contain only one intensity value per pixel and can also be referred to as a gray scale image.  MultiSpectral contains each color band as its own gray scale image.  Any operation which can process an ImageSingleBand can process one of the bands inside of a MultiSpectral image, allowing for fast implementation of some color image processing and computer vision algorithms. In the future interleaved color images will also be supported.
= Image Gray =
 
= Single Band Images =


{|
{|
|-
|-
! Class inheritance diagram for single band images
! Class Inheritance Diagram for ImageGray
|-
|-
| [[File:Image_type_single_class_diagram.png]]
| [[File:Image_type_single_class_diagram.png]]
|}
|}


Single band-images are gray scale images which represent an image using a single color or band.  They are easier to work with the multi-band data and more commonly used in computer vision because they are faster to work with.  The down side to working with gray scale images is that to create one often requires throwing away valuable color information.
Gray images are single-band images (a.k.a. gray scale images) which represent an image using a single color or band.  They are easier to work with the multi-band data and more commonly used in computer vision because they are faster to work with and much of the visual information is contained in a single band.   


== Data Structure ==
== Internal Data Structure ==


All images in BoofCV extend ImageBase and have the following data structure:
All images in BoofCV extend ImageBase and has the following structure:
<pre>
<pre>
public int startIndex;
public int startIndex;
Line 59: Line 65:
The easiest way to access a pixel is using the set() and get() functions.  However pixels can also be accessed directly.
The easiest way to access a pixel is using the set() and get() functions.  However pixels can also be accessed directly.
<pre>
<pre>
ImageFloat32 image = new ImageFloat32(60,60);
GrayF32 image = new GrayF32(60,60);


int x = 10;
int x = 10;
Line 77: Line 83:


<pre>
<pre>
ImageUInt8 image = new ImageUInt8(60,60);
GrayU8 image = new GrayU8(60,60);
image.set(x,y,45);
image.set(x,y,45);
int value = image.data[ image.startIndex + y*image.stride + x] & 0xFF;
int value = image.data[ image.startIndex + y*image.stride + x] & 0xFF;
</pre>
</pre>


= Multi Spectral Images =
= Interleaved Images =
 
 
{|
|-
! Class Inheritance Diagram for Interleaved Images
|-
| [[File:Image_type_interleaved_diagram.png]]
|}
 
 
 
= Planar Images =


MultiSpectral images are color images where each color band is stored as a separate gray scale image. Storing color images in this format is less common than the typical interleaved format, but allows color images to be easily processed using highly optimized gray scale code that is common throughout BoofCV.  The main disadvantage is that for some algorithm types an interleaved image will be faster to process when color information can not be processes in each band independently.
Planar images are color images where each color band is stored as a separate gray scale image. Storing color images in this format is less common than the typical interleaved format, but allows color images to be easily processed using highly optimized gray scale code that is common throughout BoofCV.  


Working with MultiSpectral images is easy.  To access the intensity of a pixel in different colors, simply access the image for that band then use the gray scale accessors functions.  Below a code sniplet shows how to create a multi spectral image and set the pixel at (5,5) to have a value of (255,50,50), which would appear to be red in an RGB image.
Working with Planar images is easy.  To access the intensity of a pixel in different colors, simply access the image for that band then use the gray scale accessors functions.  Below a code sniplet shows how to create a multi spectral image and set the pixel at (5,5) to have a value of (255,50,50), which would appear to be red in an RGB image.
<pre>
<pre>
MultiSpectral<ImageUInt8> image = new MultiSpectral<ImageUInt8>(ImageUInt8.class,100,200,3);
Planar<GrayU8> image = new Planar<GrayU8>(GrayU8.class,100,200,3);
image.getBand(0).set(5,5,255);
image.getBand(0).set(5,5,255);
image.getBand(1).set(5,5,50);
image.getBand(1).set(5,5,50);
Line 100: Line 118:
<pre>
<pre>
public static void main( String args[] ) {
public static void main( String args[] ) {
ImageFloat32 image = new ImageFloat32(100,100);
GrayF32 image = new GrayF32(100,100);
ImageFloat32 sub = image.subimage(10,15,30,30);
GrayF32 sub = image.subimage(10,15,30,30);


sub.set(0,0,100);
sub.set(0,0,100);
Line 117: Line 135:
Type specific code:
Type specific code:
<pre>
<pre>
public static ImageUInt8 typeSpecific( ImageUInt8 image ) {
public static GrayU8 typeSpecific( GrayU8 image ) {
ImageUInt8 output = new ImageUInt8(image.width,image.height);
GrayU8 output = new GrayU8(image.width,image.height);
BlurImageOps.gaussian(image,output,-1,2,null);
BlurImageOps.gaussian(image,output,-1,2,null);

Latest revision as of 08:15, 29 March 2016

Its not surprising that images play a key role inside of BoofCV, since its a computer vision library. Images are 2D arrays which store pixel intensity values, where a single pixel represents the output of a optical sensor. In BoofCV, to help catch more errors at compile time and to ensure more meaningful error messages, several different types of images are provided. The more standard approach (such as is done in Java2D or OpenCV) is to provide a single data type which can represent many different image types.

There is one image type for each primitive data element, including signed and unsigned data types. For example, GrayF32 and GrayU8 represent single band images containing 32-bit floats and unsigned 8-bit integer pixels, respectively. Two color image formats are supported, Planar and Interleaved in different parts of the code. Which one is supported depends on which one is the most advantageous for the algorithm.

The type of primitive data structure contained in a class is specified with a suffix. Not all data types are supported due to language limitations found in Java, e.g. no unsigned 64-bit integer.

U8
Unsigned 8-bit integer
S8
Signed 8-bit integer
U16
Unsigned 16-bit integer
S16
Signed 16-bit integer
S32
Signed 32-bit integer
S64
Signed 64-bit integer
F32
Floating point 32-bit
F64
Floating point 64-bit


Why not use BufferedImage?

Images in BoofCV are designed for high speed image processing with simple data structures for easier development. BufferedImages are highly abstracted and designed to encapsulate just about any image format. Manipulating images using BufferedImage's getRGB() and setRGB() functions is painfully slow. In-fact the only way to get any speed out of BufferedImages is to use the forbidden low level interfaces! Compared to BoofCV where its images are highly type specific, with absolutely minimal abstraction and strong typing. In BoofCV, raw pixel data can even manipulated without accessors functions, for added speed in low level functions easily.

BoofCV provides several routines for quickly converting BufferedImage into BoofCV images. Using BoofCV routines for converting to/from BufferedImages are very fast because they access low level rasters inside of BufferedImage and gracefully handle situations where access is blocked to the rasters (e.g. Applets).

Image Families

Class Inheritance Diagram for High Level Image Types
Image type class diagram.png

All image types extend the abstract ImageBase class. ImageBase proves basic information on the size, shape, and internal structure of the class. Immediate children of ImageBase are ImageGray, Interleaved, and Planar. ImageGray contain only one intensity value per pixel and can also be referred to as a gray scale image. Interleaved stores color information in adjacent elements for the same pixel. Planar contains each color band as its own gray scale image. Once advantage to Planar images is that the individual bands can be treated as ImageGray, which is exactly what they are.

Image Gray

Class Inheritance Diagram for ImageGray
Image type single class diagram.png

Gray images are single-band images (a.k.a. gray scale images) which represent an image using a single color or band. They are easier to work with the multi-band data and more commonly used in computer vision because they are faster to work with and much of the visual information is contained in a single band.

Internal Data Structure

All images in BoofCV extend ImageBase and has the following structure:

public int startIndex;
public int stride;
public int width;
public int height;
public DATA_TYPE data;

where DATA_TYPE depends upon the type of primitive data stored inside the image. 'startIndex' is the first index in 'data' that represents the first pixel in the image. 'stride' is how many pixels between rows in the image. 'width' and 'height' are the number of columns and rows in the image. 'data' is where pixel intensity values are stored. Pixel information is stored in a row-major format.

Pixel Access

The easiest way to access a pixel is using the set() and get() functions. However pixels can also be accessed directly.

GrayF32 image = new GrayF32(60,60);

int x = 10;
int y = 20;

// using accessors
image.set(x,y,45);
float value1 = image.get(x,y);

// direct access
float value2 = image.data[ image.startIndex + y*image.stride + x];

System.out.println(value1+" "+value2);

The advantage to using accessors is that they are easier to use and safe. Accessors perform bounds checking and for unsigned integer images they do the necessary bitwise operations to make sure the returned data is unsigned. Example of accessing pixel information in an unsigned integer image below:

GrayU8 image = new GrayU8(60,60);
image.set(x,y,45);
int value = image.data[ image.startIndex + y*image.stride + x] & 0xFF;

Interleaved Images

Class Inheritance Diagram for Interleaved Images
Image type interleaved diagram.png


Planar Images

Planar images are color images where each color band is stored as a separate gray scale image. Storing color images in this format is less common than the typical interleaved format, but allows color images to be easily processed using highly optimized gray scale code that is common throughout BoofCV.

Working with Planar images is easy. To access the intensity of a pixel in different colors, simply access the image for that band then use the gray scale accessors functions. Below a code sniplet shows how to create a multi spectral image and set the pixel at (5,5) to have a value of (255,50,50), which would appear to be red in an RGB image.

Planar<GrayU8> image = new Planar<GrayU8>(GrayU8.class,100,200,3);
image.getBand(0).set(5,5,255);
image.getBand(1).set(5,5,50);
image.getBand(2).set(5,5,50);

Sub-Images

A sub-image is an image which is a rectangular region inside a larger image. They are useful when only a part of an image needs to be processed, e.g. region of interest, or when only part of the image should be modified. Creating sub-image is easy, just call the subimage() function, which is part of all images. Any changes you make to a sub-image will affect the larger image too.

	public static void main( String args[] ) {
		GrayF32 image = new GrayF32(100,100);
		GrayF32 sub = image.subimage(10,15,30,30);

		sub.set(0,0,100);

		System.out.println(sub.get(0,0));
		System.out.println(image.get(10,15));
	}

This outputs "100" in both print statements because the call to set() affects the sub-image and the original image.

Generics

Generics can be used to improve the readability and abstractness of code. Images in BoofCV use Java generics to provide better type checking in abstracted code.

Type specific code:

	public static GrayU8 typeSpecific( GrayU8 image ) {
		GrayU8 output = new GrayU8(image.width,image.height);
		
		BlurImageOps.gaussian(image,output,-1,2,null);
		
		return output;
	}

Abstracted code using generics:

	public static <T extends ImageBase> T generic( T image ) {
		T output = (T)image._createNew(image.width,image.height);
		
		GBlurImageOps.gaussian(image, output, -1, 2, null);
		
		return output;
	}

One disadvantage of working with generics is that not all image type errors can be caught at runtime. In the second example above it is possible to pass in an image not supported by GBlurImageOps, which will cause a RuntimeException to be thrown. At the same time it is possible to use generics to only allow a specific type of image.