Tutorial Image Segmentation

Image Segmentation and Superpixels in BoofCV

This article provides an overview of image segmentation and superpixels in BoofCV. Image segmentation is a problem in which an image is partitioned into groups of related pixels. These pixel groups can then be used to identify objects and reduce the complexity of image processing. Superpixels are a more specific type of segmentation where the partitions form connected clusters.

The following topics will be covered:

• Thresholding
• Color histogram
• Superpixels

For a list of code examples see GitHub

Up to date as of BoofCV v0.17

Thresholding

• Global Thresholding
• 

  Input TEM particle image.

• 

  Thresholded Binary Image.

The most basic way to threshold a gray scale image is by thresholding. Thresholding works b yby assigning 1 to all pixels below/above an intensity value to 1 and the rest 0. Typically the pixels which are assigned a value of 1 are an object of interest. BoofCV provides both global and adaptive (local) thresholding capabilities. The images above demonstrate a global threshold being used on a TEM image with particles.

Global thresholding is extremely fast, but it can be unclear what a good threshold is. BoofCV provides several methods for automatically computing global thresholds based on different theories. Otsu works by minimizing the spread of background and foreground pixels. Entropy maximizes the entropy between the foreground and background regions. The image's mean can also be quickly computed. The code below computes and applies an Otsu global threshold.

GThresholdImageOps.threshold(input, binary, GThresholdImageOps.computeOtsu(input, 0, 256), true);

• 

  Calibration grid with variable lighting. Left: original, Middle: Otsu, Right: Adaptive Square

A global threshold across the whole image breaks down if the lighting conditions vary within the image. This is demonstrated in the figure above. A calibration grid is the object of interest and spotlight affect can be seen. When a global Otsu threshold (center) is applied many of the squares are lost. Instead an adaptive threshold using a square region (right) is used and the squares are clearly visible. Adaptive thresholds based on squares and Gaussian distributions work well as long as there is sufficient texture inside the local region. If there is no texture then they produce noise. The figure below shows a text sample with a complex background being binarized. The adaptive square filter produces excessive noise, while the adaptive Sauvola produces a much cleaner image.

• 

  Complex text sample. Left: Original, Middle: Adaptive Square, Right: Sauvola

After an image has been thresholded how do you extract objects from it? This can be done by finding the contour

• Binary Object Extraction
• 

  Labeled Binary Image

How do you select a threshold? Discuss...

What happens if there is no good global threshold for an image? This

Show chessboard example for local

Once segmented the next step is typically to identify individual blobs, which can be done using BLAH.

Color Histogram

Next up is color based image segmentation. In this example you can click on a pixel and it will find all pixels with a similar color. This is done by convert the image into a color space which is independent of intensity, HSV, and then selecting pixels which are similar.

One application includes identifying a road for an autonomous vehicle to drive along.

Superpixels

Superpixels have been gaining popularity recently. BoofCV contains implementations of several recently developed algorithms.

Mean-Shift SLIC Felzenszwalb-Huttenlocher Watershed

Which one is the best is highly application dependent. Here is application included with BoofCV which allows you to experiment and play with different parameters.

(go through a few examples)