Tutorial Fiducials

• Supported Fiducials
• 

  Square Binary

• 

  Square Image

• Calib target chess small.png

  Calibration Target

• 

  Cubes rendered on top of detected fiducials

In computer vision, a fiducial marker is a known object from which can be identified and its pose estimated. BoofCV provides built in support several different fiducials which can be easily printed. Applications are provided for automatically creating postscript files for the printer and a high level interface for detecting, identifying and pose estimation.

There are two types of fiducials supported in BoofCV, square and calibration targets. Square fiducials encode a pattern inside a black square box. These targets can be uniquely identified and provide a pose estimate. Calibration targets fiducials are repurposed targets used to calibrate cameras. Calibration fiducials tend to provide very accurate pose estimation when close to the camera, but can have difficulty as they move away. There are two significant disadvantage for calibration targets. 1) They don't provide a unique ID. 2) Most patterns are not fully orientation invariant. You can see the lack of rotation invariance when it suddenly flips 180 degrees.

Fiducial Summary Table

Type	Speed	Unique	Pose	Accuracy
Square Binary	Fast X	4096	Full	Good
	Robust X
Square Image	Fast X	∞	Full	Good
	Robust X
Calibration	X	1	Partial	Best Close

Speed measure on a BLAH computer.

Quick Start

1. Calibrate your camera and save results (Tutorial)
2. Print fiducial
3. Launch fiducial webcam application
4. Point camera at fiducial

For the last step you need to launch TrackFiducialWebcam in boofcv/application. Perhaps the easiest way to do that is with the following Gradle script.

If you launch it without specifying an 'intrinsic.xml' for your camera it will run and will use the calibration file my camera! This will produce poor pose estimates since the distortion, FOV and other intrinsic parameters are likely to be incorrect.

High Level Interface

The easy to use high-level interface for fiducials is FiducialDetector, see below for a skeleton. FactoryFiducial is the easiest way to create instances for different target types and it hides much of the complexity. Some detectors require additional information after construction. For example, square image fiducials require images be provided for each target it can detect. See code examples below.

public interface FiducialDetector<T extends ImageBase>
{
	public void detect( T input );

	public void setIntrinsic( IntrinsicParameters intrinsic );

	public int totalFound();

	public void getFiducialToWorld(int which, Se3_F64 fiducialToSensor );

	public int getId( int which );

	public ImageType<T> getInputType();
}

Examples:

• Square Binary Example
• Square Image Example
• Calibration Target Example

Square Fiducials

A) White outside region makes it easier to detect. B) Black square border which. C) Encoded image or pattern.

All square fiducials share a common code base. A target contains a black square of constant width and inside there is an image or pattern. The pattern is used to uniquely identify the fiducial and determine its orientation. A full 6-DOF pose is estimated from these fiducials. These targets are inspired by ARToolkit, but the code is not a port and was developed from scratched to fully utilize existing code in BoofCV.

The initial processing step is to threshold the image. BoofCV provides a various thresholding techniques for doing so. FactoryFiducial provides "robust" and "fast" techniques. Robust will use a locally adaptive algorithm which is invariant to local changes in lighting while fast uses a constant threshold. The next step is to find the contour of blobs in the image. Clearly invalid contours are pruned and a polygon fit to the contour. This contour is used to provide the initial estimate of the squares edges. An expectation-maximumization algorithm is used to fit lines to the contour and the corners are found by the intersection of the lines. Once the corners are found a homography is computed and then decomposed to return the pose.

One the pose is known perspective distortion can be used to remove and a synthetic image created. The fiducial is uniquely identified using the synthetic image. Orientation ambiguity is resolved using the fiducials pattern inside the square. For the binary pattern 4 corners are used. For the image 4 different possible orientations are considered and the best match used.

The specifics for each type of square fiducial is discussed below.

Square Binary

The square binary fiducial encodes a 12-bit number, 4096 possible values, using a binary pattern. The number is encoded by breaking up the inner portion into 16 squares in a 4x4 grid. Three of the corners are always white and one black. This is how it resolves an orientation ambiguity.

A new fiducial can be created using the DetectFiducialSquareBinary application. For easy of use a Gradle script has been provided:

gradle fiducialBinary -Pwidth=10 -Pnumber=325

:applications:classes UP-TO-DATE
:applications:fiducialBinary
Target width 10.0 (cm)  number = 325
101000101000

BUILD SUCCESSFUL

This will create a pattern which is 10cm wide and encodes the number 325. The output will be saved in "boofcv/applications/pattern.eps" file. See the top figure the resulting pattern.

Detection is easy enough using the high level Fiducial interface. See the example below for the details.

Binary Detection Example

Square Image

Square image fiducials identify a target by embedding an image inside a square. The theoretical maximum number of unique fiducials is quite large, but in practice is limited by camera resolution and processing power. The time to process an image increases linearly O(N) with the number images. For a small number of images constant time overhead (binaryization and contour identification) will dominate because images are encoded efficiently as binary numbers in integers and fast bitwise operators used to compare.

A fiducial can be created from any image. CreateFiducialSquareImageEPS is used to create new postscript fiducial files and will automatically rescale the image so that it is square and ensure that it's the correct size. For easy of use a Gradle script has been provided:

gradle fiducialImage -Pwidth=10.0 -Pimage="../data/applet/fiducial/image/dog.png"

:applications:classes UP-TO-DATE
:applications:fiducialImage
Target width 10.0 (cm)  image = dog.png

BUILD SUCCESSFUL

This will create a pattern which is 10cm wide and encodes the image contained in 'dog.png'. The output will be saved in "boofcv/applications/fiducial_image.eps" file. See the top figure the resulting pattern.

Calibration Target

The patterns used to calibrate the camera are also fiducials. These are designed for high accuracy when close to the camera and tolerant to moderate lens distortion. While quite good up close their accuracy degrades the farther away and the more acute the viewing angle is, perhaps faster than other target types.

There are two significant draw backs to using calibration targets as pose estimation fiducials. 1) They only give a partial pose estimate. 2) Only one can be visible at a time. A partial pose estimate is given because of symmetry along each axis. It is possible to choose a number of squares such that some of the symmetry goes away in one or more axis, but still requires you to be careful. When being developed it was assumed that only one is visible at a time and there is no ID encoded into the pattern. Thus if there is more than one there will be confusion.

A set of patterns created for calibration can be found in 'boofcv/data/evaluation/calibration'

A1_chess.ps   A4_chess.ps   letter_chess.ps 
A1_square.ps  A4_square.ps  letter_square.ps

where A1, A4, and letter refers to the paper size. Chess for chessboard pattern and square for square grid. See calibration tutorial for more information.

• Calibration Target Example