BoofCV - User contributions [en]

Tutorial Fiducials

2017-04-22T19:10:42Z

Orfgen: /* Square Image */ Corrected github link "applet" -> "example"

<center>
<gallery caption="Supported Fiducials" heights=200 widths=200 >
Image:Fiducial_squre_binary.png| Square Binary
Image:Fiducial_square_image.png| Square Image
Image:Calib_target_chess_small.png| Calibration Target
Image:Fiducial square binary detected.jpg| Cubes rendered on top of detected fiducials
</gallery>
</center>

In computer vision, a fiducial marker is a known object which can be easily identified. BoofCV provides built in support several different fiducials, all of which can be easily printed on paper. Applications are provided for creating your own postscript files. BoofCV library provides a single high level interface for detection, identification and pose estimation of all fiducials. Alternatively you can use the low level API and access fiducial specific information.

<center>[https://youtu.be/qJWDK_FrgHE Fiducial Video Overview]</center>

There are two types of fiducials supported in BoofCV, square and calibration targets. Square fiducials encode a large number of unique patterns inside a black square box. These targets can be uniquely identified and provide a full pose estimate. Calibration targets fiducials are repurposed targets used to calibrate cameras. Calibration fiducials tend to provide very accurate pose estimation when close to the camera, but can have difficulty as they move away. There are two significant disadvantage for calibration targets. 1) They don't provide a unique ID. 2) Most patterns are not fully orientation invariant. You can see the lack of rotation invariance when it suddenly flips 180 degrees.

<center>
'''Fiducial Summary Table'''
{| class="wikitable"
! Type !! Variant !! Speed (FPS) !! Unique !! Pose !! Accuracy
|-
|Square Binary|| Fast ||style="text-align:center;"| 175 || Varies || Full || Good
|-
| || Robust ||style="text-align:center;"| 67 || || ||
|-
|Square Image || Fast ||style="text-align:center;"| 170 || ∞ || Full || Good
|-
| || Robust ||style="text-align:center;"| 66 || || ||
|-
|Calibration || Chessboard ||style="text-align:center;"| 97 || 1 || Partial || Best Close
|-
| || Square Grid ||style="text-align:center;"| 100 || 1 || Partial || Best Close
|}
''Speed'' to detect multiple fiducials in a 640x480 image on a Intel Core i7-2600 3.4 Ghz. ''Unique'' max number of unique targets it can identify. ''Pose'' indicates if a full 6-DOF estimate is found or subset. Infinity symbol really means "lots". Binary fiducials can be configured with different inner grids. Total number of unique fiducials will very from 32 to 1.15e18 for a 3x3 or 8x8 grid, respectively.
</center>

= Quick Start =

# Calibrate your camera and save results ([[Tutorial_Camera_Calibration|Tutorial]])
#* Technically optional, but highly recommended
# Print binary fiducial, e.g. [http://boofcv.org/notwiki/fiducials/square0643.eps Binary #0643]
# Launch fiducial webcam application
# Point camera at fiducial

== Application ==

To help you get started and quickly test ideas there's a fiducial application included with BoofCV. It will detect fiducials, then draw a 1/2 cube on top of them so you can see how accurate its 3D estimate is. To run the application do the following:

<syntaxhighlight lang="bash">
cd boofcv/applications
gradle applicationsJar
java -jar applications.jar WebcamTrackFiducial --Resolution=640:480 BINARY
</syntaxhighlight>

If you calibrated your camera you can do the following and get better results:

<syntaxhighlight lang="bash">
java -jar applications.jar WebcamTrackFiducial --Intrinsic=intrinsic.xml BINARY
</syntaxhighlight>

To get a list of commands and see how to track other types of fiducials just enter the command with no arguments.
<syntaxhighlight lang="bash">
java -jar applications.jar WebcamTrackFiducial
</syntaxhighlight>

== Printable Fiducials ==

The following is a printable documents for all the types of fiducials supported in BoofCV. Print these to get started quickly, but creating your own is also easy.

{| class="wikitable"
! Square Binary !! Chessboard
|-
|
* [http://boofcv.org/notwiki/fiducials/square0643.eps File 4x4 Binary #0643]
* [http://boofcv.org/notwiki/fiducials/square0284.eps File 4x4 Binary #0284]
|
* [http://boofcv.org/notwiki/calibration/letter_chess.ps Letter Sized Paper: Chessboard, 5 by 7, 30mm Squares]
* [http://boofcv.org/notwiki/calibration/A4_chess.ps A4 Sized Paper: Chessboard, 5 by 8, 30mm Squares]
* [http://boofcv.org/notwiki/calibration/A1_chess.ps A1 Sized Paper: Chessboard, 8 by 12, 60mm Squares]
|-
! Square Image !! Square grid
|-
|
* [http://boofcv.org/notwiki/fiducials/10cm_pentarose.eps Pentarose]
* [http://boofcv.org/notwiki/fiducials/10cm_yinyang.eps Yin Yang]
* [http://boofcv.org/notwiki/fiducials/all_a4.eps Sheet of All, A4]
* [http://boofcv.org/notwiki/fiducials/all_letter.eps Sheet of All, Letter]
* [https://github.com/lessthanoptimal/BoofCV-Data/tree/master/applet/fiducial/image/eps bunch more]
|
* [http://boofcv.org/notwiki/calibration/letter_square.ps Letter Sized Paper: Square Grid, 3 by 4, 30mm Squares]
* [http://boofcv.org/notwiki/calibration/A4_square.ps A4 Sized Paper: Square Grid, 3 by 5, 30mm Squares]
* [http://boofcv.org/notwiki/calibration/A1_square.ps A1 Sized Paper: Square Grid, 4 by 6 60mm Squares]
|}

= Creating your Own Fiducial =

For square fiducials, a convenient command-line application is provided which can create printable postscript (.ps) documents which contain one or more fiducials on them. For calibration targets, prefabricated patterns are provided which can also be printed.

== Square Binary ==
Fiducials can be made using the applications.jar you created earlier. Usual for complete instructions just enter in the classes name with no arguments.

<syntaxhighlight lang="bash">
java -jar applications.jar CreateFiducialSquareBinary -OutputFile=fiducial.ps -PrintInfo -Units=cm 12 284
</syntaxhighlight>

That will create a printable fiducial.ps file that encodes the number 284 in a 4x4 square that's 12 centimeters. The number of elements which compose its inner grid and the width of the outside border are both configurable. By default all code assumes a 4x4 grid and 0.25 fractional border width. This produces a reasonable compose between detection robustness and the number of features it can encode, which is 4096.

== Square Image ==

A fiducial can be easily created from any image using "applications.jar".

<syntaxhighlight lang="bash">
java -jar applications.jar CreateFiducialSquareImage -OutputFile=fiducial.ps -PrintInfo -Units=cm 12 pentarose.png
</syntaxhighlight>
This will create a pattern which is 12cm wide and encodes the image contained in 'pentarose.png'. The output will be saved in "fiducial.ps" file.

Not feeling very creative or just want to see some example images? Several patterns are contained in [https://github.com/lessthanoptimal/BoofCV-Data/tree/master/example/fiducial/image/patterns "data/example/fiducial/image/patterns/"].

= How Do Square Fiducials Work? =
[[File:Square fiducial parts.png|thumb|300px|The origin of the fiducial is at its center with x and y axises as shown. The fiducial itself can be divided into three regions, a) white outside region which provides contrast against the black border, b) black square border, and c) encoded image or pattern.]]

All square fiducials share a common code base. A target contains a black square of constant width and inside there is an image or pattern. The pattern is used to uniquely identify the fiducial and determine its orientation. A full 6-DOF pose is estimated from these fiducials. These targets are inspired by ARToolkit, but the code is not a port and was developed from scratched.

'''Processing Steps'''
# Create binary image by thresholding
# Contours of dark objects
# Contour to crude quadrilateral
# Sub-pixel refinement of quadrilateral
# Undistort quadrilateral image and match to known patterns
# Extract 3D pose of fiducial

''Step 1:'' [[Example_Thresholding|Thresholding]] is performed using either a user configurable fixed threshold or locally adaptive threshold, fast and robust respectively. While slower robust can handle variations in lighting much better.

''Step 2:'' Contours are found from the binary blobs using [http://boofcv.org/javadoc/boofcv/alg/filter/binary/LinearContourLabelChang2004.html Chang 2004].

''Step 3:'' The contour is converted into a polygon using a [[Example_Fit_Polygon|split and merge]] technique. Only four sided polygons are accepted.

''Step 4:'' The crude polygons are refined to subpixel accuracy. This is done by iteratively refining the estimate of each side's line. That is done by computing the difference of line integrals tangential to each line, then using their intensity values to compute a weighted line.

''Step 5:'' The four corners are now known to a high level of precision, these are then used to compute a homography which is then used to generate an undistorted square image of the inner pattern. This pattern is then used by the specific algorithm to identify a known target or compute a number. Noisy images are rejected here.

''Step 6:'' The pose is computed through a combination of [[Example_PnP|P3P and P4P]] techniques. The one with the smallest residual error is selected. Then non-linear refinement is performed. Care is taken to reduce the effects of head on small fiducials, which have a degenerate geometry. In that situation significant changes in orientation result in only a small change on the image. That is handled by placing a greater emphasis on reprojection error caused by orientation.

NOTE: Steps 2 to 4 are carried out by the [[Example_Detect_Black_Polygons|black polygon detector]].

== Square Binary ==

The square binary fiducial encodes a 12-bit number, 4096 possible values, using a binary pattern. The number is encoded by breaking up the inner portion into 16 squares in a 4x4 grid. Three of the corners are always white and one black. This is how it resolves an orientation ambiguity.

== Square Image ==

When an image is loaded into this type of fiducial it is first converted into a square image then down sampled into a low resolution image and encoded efficiently . When processing a video feed and a fiducial is detected the pattern is undistorted as usual. Then the hamming distance between the just observed pattern and all the known patterns is found. The pattern with the best score within tolerance is accepted.

= Programming =

All of these different types of fiducials can be used through a high level interface, *FiducialDetector*. FactoryFiducial is the easiest way to create instances of different fiducial types and it hides much of the complexity. Some detectors require additional information after construction. For example, square image fiducials require images be provided for each target it can detect. A sketch of how to process a single image is shown below.

<syntaxhighlight lang="java">
FiducialDetector<ImageFloat32> detector = FactoryFiducial.pickAFiducial(...);
... additional fiducial specific configuration goes here ...
detector.setIntrinsic(param);
detector.detect(image);
Se3_F64 targetToSensor = new Se3_F64();
for (int i = 0; i < detector.totalFound(); i++){
System.out.println("Target ID = "+detector.getId(i));
System.out.println("Target width = "+detector.getWidth(i));
detector.getFiducialToWorld(i,targetToSensor);
System.out.println("Location:");
}
</syntaxhighlight>

See the examples below for a more understanding of how to use these different types of fiducials.

Applets:
* [[Applet_Fiducials| Applet Fiducials]]

Examples:
* [[Example Fiducial Square Binary| Square Binary Example]]
* [[Example Fiducial Square Image| Square Image Example]]
* [[Example_Calibration_Target_Pose|Calibration Target Example]]

Talk:Tutorial Quick Start

2017-04-21T21:09:36Z

Orfgen: Added short list of methods that do not exist in 0.26, but are given as example here.

These methods from the filter section do not exist anymore:
FactoryImageBorder.extend(input);
GImageDerivativeOps.sobel(blurred, derivX, derivY, BorderType.EXTENDED);
AnyImageDerivative<T,D> deriv = GImageDerivativeOps.createDerivatives((Class<T>)input.getClass(),derivType);

Tutorial Quick Start

2017-04-21T21:02:38Z

Orfgen: /* Filters */ createImage -> createSingleBand

The following tutorial is intended to provide just enough information for you to quickly set up and start development with BoofCV. If you are not familiar with the [http://java.oracle.com Java programming language] or its associated development tools, you must fix that first because BoofCV is written entirely in Java. It is highly recommended that you use a tool like [https://gradle.org Gradle] to build your own project and have it download the jars for you. If you enjoy doing things the slow and tedious way we are there for you and provide all the jars.

== Step One: Obtaining ==

The first step in using BoofCV is either adding it to your dependency list, downloading the precompiled Jars, or building it from Source.

Latest Official Release:
* Source Code [[Download:BoofCV|Download Page]]
* Jars [[Download:BoofCV|Download Page]]
* Maven and Gradle [[Download:BoofCV|Download Page]]

== Step Two: Running Examples and Demonstrations ==

Examples are short pieces of code which are designed to be easy to understand and show you how to perform some task. Demonstrations are more complex applications which visualize different aspects of an algorithm. The code for a demonstration is not designed to be easy to learn from and can be quite complex due to its integration with a GUI.

Source Code:
* [https://github.com/lessthanoptimal/BoofCV/tree/master/examples boofcv/examples]
* [https://github.com/lessthanoptimal/BoofCV/tree/master/demonstrations boofcv/demonstrations]

The easiest way to run an example or demonstration is to launch their respective master applications. You can also load up the source code in your favorite IDE and run the applications directly.

<syntaxhighlight lang="bash">
cd boofcv
./gradlew examples
java -jar examples/examples.jar
./gradlew demonstrations
java -jar demonstrations/demonstrations.jar
</syntaxhighlight>

= HELP ME!! =

Having trouble or have a suggestion? Post a message on the BoofCV message board! Don't worry it's a friendly place.
* [http://groups.google.com/group/boofcv Message Board]

= Quick Reference =

The remainder of this tutorial is intended to act as a quick reference of low level image processing routines in BoofCV.

{| class="wikitable"
! Term !! definition
|-
| single band || The image supports only one color
|-
| floating point || Image elements are of type float or double
|-
| unsigned || Image elements can only be positive integers
|-
| signed || Image elements can be either positive or negative integers
|-
| generics || Allows strong typing in abstracted code. Introduced in Java 1.5. Click [http://docs.oracle.com/javase/tutorial/java/generics/index.html here].
|}

== The Basics ==

BoofCV supports 3 types of images; Gray (single band images), Planar (multi-band in a planar format), and Interleaved (traditional multi-band image format). The first two, gray and planar are fully supported while interleaved is partially supported. Gray and planar images are just easier to work with most of the time which is why they are fully supported. Interleaved is only supported where there is a performance advantage that was significant.

<syntaxhighlight lang="java">
GrayU8 image = new GrayU8(100,150);
</syntaxhighlight>
Creating an unsigned 8-bit integer single band image with width=100 and height=150.

<syntaxhighlight lang="java">
GrayF32 image = new GrayF32(100,150);
</syntaxhighlight>
Creating a floating point single band image with width=100 and height=150.

<syntaxhighlight lang="java">
Planar<GrayU8> image = new Planar<GrayU8>(GrayU8.class,100,200,3);
</syntaxhighlight>
Creates a color planar image with 3 bands using GrayU8 for each band.

<syntaxhighlight lang="java">
GrayF32 image = UtilImageIO.loadImage("test.png",GrayF32.class);
</syntaxhighlight>
Loads a single band image of type GrayF32 from a file.

<syntaxhighlight lang="java">
public static <T extends ImageBase> T generic( Class<T> imageType ) {
T image = UtilImageIO.loadImage("test.png",imageType);
</syntaxhighlight>
Loads an image with the specified type inside a function that uses Java generics.

<syntaxhighlight lang="java">
BufferedImage out = ConvertBufferedImage.convertTo(image,null);
</syntaxhighlight>
Converts an image into a BufferedImage to provide better integration with Java2D (display/saving). Pixel values must be in the range of 0 to 255.

<syntaxhighlight lang="java">
BufferedImage out = VisualizeImageData.grayMagnitude(derivX,null,-1);
</syntaxhighlight>
Renders a signed single band image into a gray intensity image.

<syntaxhighlight lang="java">
BufferedImage out = VisualizeImageData.colorizeSign(derivX,null,-1);
</syntaxhighlight>
Renders a signed single band image into a color intensity image.

<syntaxhighlight lang="java">
BufferedImage out = ConvertBufferedImage.convertTo(image,null);
ShowImages.showWindow(out,"Output");
</syntaxhighlight>
Displays an image in a window using Java swing.

== Pixel Access ==

The image type must be known to access pixel information. The following show how to access pixels for different image types. For more information on the image data structure and direct access to the raw data array see [[Tutorial Images]] for more details

<syntaxhighlight lang="java">
public static void function( GrayF32 image )
{
float pixel = image.get(5,23);
image.set(5,23,50.3);
</syntaxhighlight>
Gets and sets the pixel at (5,23). Note that set() and get() functions are image type specific. In other words, you can't access pixel without knowing the image type.

<syntaxhighlight lang="java">
public static void function( GrayU8 image )
{
int pixel = image.get(5,23);
image.set(5,23,50);
</syntaxhighlight>
Similar to the above example but for an 8-bit unsigned integer image. Note the image.get() returns 'int' and not 'byte'.

<syntaxhighlight lang="java">
public static void function( GrayI image )
{
int pixel = image.get(5,23);
image.set(5,23,50);
</syntaxhighlight>
In fact the same code will work for all integer images, except SInt64 which uses longs and not ints. Internally UInt8 stores its pixels as a byte array, but set() and get() return int because Java internally does not use bytes on the register.

<syntaxhighlight lang="java">
public static void function( Planar<GrayU8> image )
{
int pixel = image.getBand(0).get(5,23);
image.getBand(0).set(5,23,50);
</syntaxhighlight>
Planar images are essentially arrays of ImageGray. To set or get a pixel value first access the particular band that needs to be changed then use the standard accessors inside of ImageSingleBand.

== Filters ==

<syntaxhighlight lang="java">
public static void procedural( GrayU8 input )
{
GrayU8 blurred = new GrayU8(input.width,input.height);
BlurImageOps.gaussian(input,blurred,-1,blurRadius,null);
</syntaxhighlight>
Applies Gaussian blur to an image using a type specific procedural interface.

<syntaxhighlight lang="java">
public static <T extends ImageGray, D extends ImageGray>
void generalized( T input )
{
Class<T> inputType = (Class<T>)input.getClass();

T blurred = GeneralizedImageOps.createSingleBand(inputType,input.width, input.height);
GBlurImageOps.gaussian(input, blurred, -1, blurRadius, null);
</syntaxhighlight>
Applies Gaussian blur to an image using an abstracted procedural interface. Note the G in front of BlurImageOps that indicates
it contains generic functions.

<syntaxhighlight lang="java">
public static <T extends ImageGray, D extends ImageGray>
void filter( T input )
{
Class<T> inputType = (Class<T>)input.getClass();
T blurred = GeneralizedImageOps.createSingleBand(inputType, input.width, input.height);
BlurFilter<T> filterBlur = FactoryBlurFilter.gaussian(inputType, -1, blurRadius);
filterBlur.process(input,blurred);
</syntaxhighlight>
Creates an image filter class for computing the Gaussian blur. Provides greater abstraction.

<syntaxhighlight lang="java">
// type specific sobel
GradientSobel.process(blurred, derivX, derivY, FactoryImageBorder.extend(input));
// generic
GImageDerivativeOps.sobel(blurred, derivX, derivY, BorderType.EXTENDED);
// filter
ImageGradient<T,D> gradient = FactoryDerivative.sobel(inputType, derivType);
gradient.process(blurred,derivX,derivY);
</syntaxhighlight>
Three ways to compute the image gradient using a Sobel kernel.

<syntaxhighlight lang="java">
public static <T extends ImageGray, D extends ImageGray>
void example( T input , Class<D> derivType ) {
AnyImageDerivative<T,D> deriv = GImageDerivativeOps.createDerivatives((Class<T>)input.getClass(),derivType);

deriv.setInput(input);
D derivX = deriv.getDerivative(true);
D derivXXY = deriv.getDerivative(true,true,false);
</syntaxhighlight>
Useful class for computing arbitrary image derivatives. Computes 1st order x-derive and then 3rd order xxy derivative.

== Binary Images ==

<syntaxhighlight lang="java">
ThresholdImageOps.threshold(image, binary, 23, true);
</syntaxhighlight>
Creates a binary image by thresholding the input image. Binary must be of type GrayU8.

<syntaxhighlight lang="java">
binary = BinaryImageOps.erode8(binary, 1, null);
</syntaxhighlight>
Apply an erode operation once on the binary image, writing over the original image reference.

<syntaxhighlight lang="java">
BinaryImageOps.erode8(binary, 1, output);
</syntaxhighlight>
Apply an erode operation once on the binary image, saving results to the output binary image.

<syntaxhighlight lang="java">
BinaryImageOps.erode4(binary, 1, output);
</syntaxhighlight>
Apply an erode operation once with a 4-connect rule.

<syntaxhighlight lang="java">
int numBlobs = BinaryImageOps.contour(binary, ConnectRule.FOUR, blobs).size();
</syntaxhighlight>
Detect and label blobs in the binary image using a 4-connect rule. blobs is an image of type GrayS32.

<syntaxhighlight lang="java">
BufferedImage visualized = VisualizeBinaryData.renderLabeled(blobs, numBlobs, null);
</syntaxhighlight>
Renders the detected blobs in a colored image.

<syntaxhighlight lang="java">
BufferedImage visualized = VisualizeBinaryData.renderBinary(binary, false, null);
</syntaxhighlight>
Renders the binary image as a black white image. false means the colors are not inverted.

== Suggested Hardware ==

To do computer vision you need a camera. Here's a list of recommended cameras

* [http://amzn.to/2igJ9GC Webcam: Logitech C920]
* [http://amzn.to/2hWGohn 360 Camera: Theta S ]

Webcams are great for basically everything but structure from motion (SFM) applications. Their images often look better than much more expensive scientific cameras. Unfortunately they have a rolling shutter which breaks SFM algorithms if anything in the scene or the camera is moving.

The Theta S is a 360 camera composed of two fisheye cameras. Interestingly it is one of the few consumer grade cameras to provide a global shutter! Making it useful for SFM applications.

(The above links are Amazon affiliate. If you do plan on purchasing one of those cameras please help finance BoofCV and click on those links.)

Tutorial Quick Start

2017-04-21T20:44:09Z

Orfgen: Changed Binary Images code to 0.26

The following tutorial is intended to provide just enough information for you to quickly set up and start development with BoofCV. If you are not familiar with the [http://java.oracle.com Java programming language] or its associated development tools, you must fix that first because BoofCV is written entirely in Java. It is highly recommended that you use a tool like [https://gradle.org Gradle] to build your own project and have it download the jars for you. If you enjoy doing things the slow and tedious way we are there for you and provide all the jars.

== Step One: Obtaining ==

The first step in using BoofCV is either adding it to your dependency list, downloading the precompiled Jars, or building it from Source.

Latest Official Release:
* Source Code [[Download:BoofCV|Download Page]]
* Jars [[Download:BoofCV|Download Page]]
* Maven and Gradle [[Download:BoofCV|Download Page]]

== Step Two: Running Examples and Demonstrations ==

Examples are short pieces of code which are designed to be easy to understand and show you how to perform some task. Demonstrations are more complex applications which visualize different aspects of an algorithm. The code for a demonstration is not designed to be easy to learn from and can be quite complex due to its integration with a GUI.

Source Code:
* [https://github.com/lessthanoptimal/BoofCV/tree/master/examples boofcv/examples]
* [https://github.com/lessthanoptimal/BoofCV/tree/master/demonstrations boofcv/demonstrations]

The easiest way to run an example or demonstration is to launch their respective master applications. You can also load up the source code in your favorite IDE and run the applications directly.

<syntaxhighlight lang="bash">
cd boofcv
./gradlew examples
java -jar examples/examples.jar
./gradlew demonstrations
java -jar demonstrations/demonstrations.jar
</syntaxhighlight>

= HELP ME!! =

Having trouble or have a suggestion? Post a message on the BoofCV message board! Don't worry it's a friendly place.
* [http://groups.google.com/group/boofcv Message Board]

= Quick Reference =

The remainder of this tutorial is intended to act as a quick reference of low level image processing routines in BoofCV.

{| class="wikitable"
! Term !! definition
|-
| single band || The image supports only one color
|-
| floating point || Image elements are of type float or double
|-
| unsigned || Image elements can only be positive integers
|-
| signed || Image elements can be either positive or negative integers
|-
| generics || Allows strong typing in abstracted code. Introduced in Java 1.5. Click [http://docs.oracle.com/javase/tutorial/java/generics/index.html here].
|}

== The Basics ==

BoofCV supports 3 types of images; Gray (single band images), Planar (multi-band in a planar format), and Interleaved (traditional multi-band image format). The first two, gray and planar are fully supported while interleaved is partially supported. Gray and planar images are just easier to work with most of the time which is why they are fully supported. Interleaved is only supported where there is a performance advantage that was significant.

<syntaxhighlight lang="java">
GrayU8 image = new GrayU8(100,150);
</syntaxhighlight>
Creating an unsigned 8-bit integer single band image with width=100 and height=150.

<syntaxhighlight lang="java">
GrayF32 image = new GrayF32(100,150);
</syntaxhighlight>
Creating a floating point single band image with width=100 and height=150.

<syntaxhighlight lang="java">
Planar<GrayU8> image = new Planar<GrayU8>(GrayU8.class,100,200,3);
</syntaxhighlight>
Creates a color planar image with 3 bands using GrayU8 for each band.

<syntaxhighlight lang="java">
GrayF32 image = UtilImageIO.loadImage("test.png",GrayF32.class);
</syntaxhighlight>
Loads a single band image of type GrayF32 from a file.

<syntaxhighlight lang="java">
public static <T extends ImageBase> T generic( Class<T> imageType ) {
T image = UtilImageIO.loadImage("test.png",imageType);
</syntaxhighlight>
Loads an image with the specified type inside a function that uses Java generics.

<syntaxhighlight lang="java">
BufferedImage out = ConvertBufferedImage.convertTo(image,null);
</syntaxhighlight>
Converts an image into a BufferedImage to provide better integration with Java2D (display/saving). Pixel values must be in the range of 0 to 255.

<syntaxhighlight lang="java">
BufferedImage out = VisualizeImageData.grayMagnitude(derivX,null,-1);
</syntaxhighlight>
Renders a signed single band image into a gray intensity image.

<syntaxhighlight lang="java">
BufferedImage out = VisualizeImageData.colorizeSign(derivX,null,-1);
</syntaxhighlight>
Renders a signed single band image into a color intensity image.

<syntaxhighlight lang="java">
BufferedImage out = ConvertBufferedImage.convertTo(image,null);
ShowImages.showWindow(out,"Output");
</syntaxhighlight>
Displays an image in a window using Java swing.

== Pixel Access ==

The image type must be known to access pixel information. The following show how to access pixels for different image types. For more information on the image data structure and direct access to the raw data array see [[Tutorial Images]] for more details

<syntaxhighlight lang="java">
public static void function( GrayF32 image )
{
float pixel = image.get(5,23);
image.set(5,23,50.3);
</syntaxhighlight>
Gets and sets the pixel at (5,23). Note that set() and get() functions are image type specific. In other words, you can't access pixel without knowing the image type.

<syntaxhighlight lang="java">
public static void function( GrayU8 image )
{
int pixel = image.get(5,23);
image.set(5,23,50);
</syntaxhighlight>
Similar to the above example but for an 8-bit unsigned integer image. Note the image.get() returns 'int' and not 'byte'.

<syntaxhighlight lang="java">
public static void function( GrayI image )
{
int pixel = image.get(5,23);
image.set(5,23,50);
</syntaxhighlight>
In fact the same code will work for all integer images, except SInt64 which uses longs and not ints. Internally UInt8 stores its pixels as a byte array, but set() and get() return int because Java internally does not use bytes on the register.

<syntaxhighlight lang="java">
public static void function( Planar<GrayU8> image )
{
int pixel = image.getBand(0).get(5,23);
image.getBand(0).set(5,23,50);
</syntaxhighlight>
Planar images are essentially arrays of ImageGray. To set or get a pixel value first access the particular band that needs to be changed then use the standard accessors inside of ImageSingleBand.

== Filters ==

<syntaxhighlight lang="java">
public static void procedural( GrayU8 input )
{
GrayU8 blurred = new GrayU8(input.width,input.height);
BlurImageOps.gaussian(input,blurred,-1,blurRadius,null);
</syntaxhighlight>
Applies Gaussian blur to an image using a type specific procedural interface.

<syntaxhighlight lang="java">
public static <T extends ImageGray, D extends ImageGray>
void generalized( T input )
{
Class<T> inputType = (Class<T>)input.getClass();

T blurred = GeneralizedImageOps.createImage(inputType,input.width, input.height);
GBlurImageOps.gaussian(input, blurred, -1, blurRadius, null);
</syntaxhighlight>
Applies Gaussian blur to an image using an abstracted procedural interface. Note the G in front of BlurImageOps that indicates
it contains generic functions.

<syntaxhighlight lang="java">
public static <T extends ImageGray, D extends ImageGray>
void filter( T input )
{
Class<T> inputType = (Class<T>)input.getClass();
T blurred = GeneralizedImageOps.createImage(inputType, input.width, input.height);
BlurFilter<T> filterBlur = FactoryBlurFilter.gaussian(inputType, -1, blurRadius);
filterBlur.process(input,blurred);
</syntaxhighlight>
Creates an image filter class for computing the Gaussian blur. Provides greater abstraction.

<syntaxhighlight lang="java">
// type specific sobel
GradientSobel.process(blurred, derivX, derivY, FactoryImageBorder.extend(input));
// generic
GImageDerivativeOps.sobel(blurred, derivX, derivY, BorderType.EXTENDED);
// filter
ImageGradient<T,D> gradient = FactoryDerivative.sobel(inputType, derivType);
gradient.process(blurred,derivX,derivY);
</syntaxhighlight>
Three ways to compute the image gradient using a Sobel kernel.

<syntaxhighlight lang="java">
public static <T extends ImageGray, D extends ImageGray>
void example( T input , Class<D> derivType ) {
AnyImageDerivative<T,D> deriv = GImageDerivativeOps.createDerivatives((Class<T>)input.getClass(),derivType);

deriv.setInput(input);
D derivX = deriv.getDerivative(true);
D derivXXY = deriv.getDerivative(true,true,false);
</syntaxhighlight>
Useful class for computing arbitrary image derivatives. Computes 1st order x-derive and then 3rd order xxy derivative.

== Binary Images ==

<syntaxhighlight lang="java">
ThresholdImageOps.threshold(image, binary, 23, true);
</syntaxhighlight>
Creates a binary image by thresholding the input image. Binary must be of type GrayU8.

<syntaxhighlight lang="java">
binary = BinaryImageOps.erode8(binary, 1, null);
</syntaxhighlight>
Apply an erode operation once on the binary image, writing over the original image reference.

<syntaxhighlight lang="java">
BinaryImageOps.erode8(binary, 1, output);
</syntaxhighlight>
Apply an erode operation once on the binary image, saving results to the output binary image.

<syntaxhighlight lang="java">
BinaryImageOps.erode4(binary, 1, output);
</syntaxhighlight>
Apply an erode operation once with a 4-connect rule.

<syntaxhighlight lang="java">
int numBlobs = BinaryImageOps.contour(binary, ConnectRule.FOUR, blobs).size();
</syntaxhighlight>
Detect and label blobs in the binary image using a 4-connect rule. blobs is an image of type GrayS32.

<syntaxhighlight lang="java">
BufferedImage visualized = VisualizeBinaryData.renderLabeled(blobs, numBlobs, null);
</syntaxhighlight>
Renders the detected blobs in a colored image.

<syntaxhighlight lang="java">
BufferedImage visualized = VisualizeBinaryData.renderBinary(binary, false, null);
</syntaxhighlight>
Renders the binary image as a black white image. false means the colors are not inverted.

== Suggested Hardware ==

To do computer vision you need a camera. Here's a list of recommended cameras

* [http://amzn.to/2igJ9GC Webcam: Logitech C920]
* [http://amzn.to/2hWGohn 360 Camera: Theta S ]

Webcams are great for basically everything but structure from motion (SFM) applications. Their images often look better than much more expensive scientific cameras. Unfortunately they have a rolling shutter which breaks SFM algorithms if anything in the scene or the camera is moving.

The Theta S is a 360 camera composed of two fisheye cameras. Interestingly it is one of the few consumer grade cameras to provide a global shutter! Making it useful for SFM applications.

(The above links are Amazon affiliate. If you do plan on purchasing one of those cameras please help finance BoofCV and click on those links.)