Tutorial Fiducials

• Supported Fiducials
• 

  Square Binary

• 

  Square Image

• Calib target chess small.png

  Calibration Target

• 

  Cubes rendered on top of detected fiducials

FIDUCIAL STABILITY AND ACCURACY WAS IMPROVED BY A LARGE MARGIN IN BOOFCV v0.19

In computer vision, a fiducial marker is a known object which can be identified and have its pose estimated. BoofCV provides built in support several different fiducials, which can be easily printed. Applications are provided for automatically creating printable postscript files and a high level interface for detection, identification and pose estimation.

There are two types of fiducials supported in BoofCV, square and calibration targets. Square fiducials encode a pattern inside a black square box. These targets can be uniquely identified and provide a full pose estimate. Calibration targets fiducials are repurposed targets used to calibrate cameras. Calibration fiducials tend to provide very accurate pose estimation when close to the camera, but can have difficulty as they move away. There are two significant disadvantage for calibration targets. 1) They don't provide a unique ID. 2) Most patterns are not fully orientation invariant. You can see the lack of rotation invariance when it suddenly flips 180 degrees.

Fiducial Summary Table

Type	Variant	Speed (FPS)	Unique	Pose	Accuracy
Square Binary	Fast	175	4096	Full	Good
	Robust	67
Square Image	Fast	170	∞	Full	Good
	Robust	66
Calibration	Chessboard	97	1	Partial	Best Close[0]
	Square Grid	100	1	Partial	Best Close[0]

Speed to detect multiple fiducials in a 640x480 image on a Intel Core i7-2600 3.4 Ghz. Unique max number of unique targets it can identify. Pose indicates if a full 6-DOF estimate is found or subset. Infinity symbol really means "lots".

[0] There is a bug in v0.19 and before where when viewed at acute angles its accuracy goes down. This has been fixed in 0.20-SNAPSHOT already.

Quick Start

1. Calibrate your camera and save results (Tutorial)
   • Technically optional, but highly recommended
2. Print binary fiducial, e.g. Binary #0643
3. Launch fiducial webcam application
4. Point camera at fiducial

Application

To help you get started and quickly test ideas there's a fiducial application included with BoofCV. It will detect fiducials, then draw a 1/2 cube on top of them so you can see how accurate its 3D estimate is. To run the application do the following:

cd boofcv/applications
gradle applciationsJar
java -cp applications.jar boofcv.app.WebcamTrackFiducial --Resolution=640:480 BINARY

If you calibrated your camera you can do the following and get better results:

java -cp applications.jar boofcv.app.WebcamTrackFiducial --Intrinsic=intrinsic.xml BINARY

To get a list of commands and see how to track other types of fiducials just enter the command with no arguments.

java -cp applications.jar boofcv.app.WebcamTrackFiducial

Printable Fiducials

The following is a printable documents for all the types of fiducials supported in BoofCV. Print these to get started quickly, but creating your own is also easy.

• Square Binary
  1. File Binary #0643
  2. File Binary #0284
• Square Image
  1. Pentarose
  2. Yin Yang
  3. Sheet of All, A4
  4. Sheet of All, Letter
  5. bunch more
• Calibration Chessboard
  1. Letter Sized Paper: Chessboard, 5 by 7, 30mm Squares
  2. A4 Sized Paper: Chessboard, 5 by 8, 30mm Squares
  3. A1 Sized Paper: Chessboard, 8 by 12, 60mm Squares
• Calibration Square Grid
  1. Letter Sized Paper: Square Grid, 5 by 7, 30mm Squares
  2. A4 Sized Paper: Square Grid, 5 by 9, 30mm Squares
  3. A1 Sized Paper: Square Grid, 7 by 11 60mm Squares

Creating your Own Fiducial

For square fiducials, a convenient command-line application is provided which can create printable EPS documents which contain one or more fiducials on them. For calibration targets, prefabricated patterns are provided which can also be printed.

Square Binary

Fiducials can be made using the applications.jar you created earlier. Usual for complete instructions just enter in the classes name with no arguments.

java -cp applications.jar boofcv.app.CreateFiducialSquareBinaryEPS -OutputFile=fiducial.eps -PrintInfo -Units=cm 12 284

That will create a printable fiducial.eps file that encodes the number 284 in a square that's 12 centimeters.

Square Image

A fiducial can be easily created from any image using "applications.jar".

java -cp applications.jar boofcv.app.CreateFiducialSquareImageEPS -OutputFile=fiducial.eps -PrintInfo -Units=cm 12 pentarose.png

This will create a pattern which is 12cm wide and encodes the image contained in 'pentarose.png'. The output will be saved in "fiducial.eps" file.

Not feeling very creative or just want to see some example images? Several patterns are contained in "data/applet/fiducial/image/patterns/".

How Do Square Fiducials Work?

The origin of the fiducial is at its center with x and y axises as shown. The fiducial itself can be divided into three regions, a) white outside region which provides contrast against the black border, b) black square border, and c) encoded image or pattern.

All square fiducials share a common code base. A target contains a black square of constant width and inside there is an image or pattern. The pattern is used to uniquely identify the fiducial and determine its orientation. A full 6-DOF pose is estimated from these fiducials. These targets are inspired by ARToolkit, but the code is not a port and was developed from scratched.

Processing Steps

1. Create binary image by thresholding
2. Contours of dark objects
3. Contour to crude quadrilateral
4. Sub-pixel refinement of quadrilateral
5. Undistort quadrilateral image and match to known patterns
6. Extract 3D pose of fiducial

Step One: Thresholding is performed using either a user configurable fixed threshold or locally adaptive threshold, fast and robust respectively. While slower robust can handle variations in lighting much better.

Step Two: Contours are found from the binary blobs using Chang 2004.

Step Three: The contour is converted into a polygon using a split and merge technique. Only four sided polygons are accepted.

Step Four: The crude polygons are refined to subpixel accuracy. This is done by iteratively refining the estimate of each side's line. That is done by computing the difference of line integrals tangential to each line, then using their intensity values to compute a weighted line.

Step Five: The four corners are now known to a high level of precision, these are then used to compute a homography which is then used to generate an undistorted square image of the inner pattern. This pattern is then used by the specific algorithm to identify a known target or compute a number. Noisy images are rejected here.

Step Six: The pose is computed through a combination of P3P and P4P techniques. The one with the smallest residual error is selected. Then non-linear refinement is performed. Care is taken to reduce the effects of head on small fiducials, which have a degenerate geometry. In that situation significant changes in orientation result in only a small change on the image. That is handled by placing a greater emphasis on reprojection error caused by orientation.

Square Binary

The square binary fiducial encodes a 12-bit number, 4096 possible values, using a binary pattern. The number is encoded by breaking up the inner portion into 16 squares in a 4x4 grid. Three of the corners are always white and one black. This is how it resolves an orientation ambiguity.

Square Image

When an image is loaded into this type of fiducial it is first converted into a square image then down sampled into a low resolution image and encoded efficiently . When processing a video feed and a fiducial is detected the pattern is undistorted as usual. Then the hamming distance between the just observed pattern and all the known patterns is found. The pattern with the best score within tolerance is accepted.

Programming

All of these different types of fiducials can be used through a high level interface, *FiducialDetector*. FactoryFiducial is the easiest way to create instances of different fiducial types and it hides much of the complexity. Some detectors require additional information after construction. For example, square image fiducials require images be provided for each target it can detect. A sketch of how to process a single image is shown below.

FiducialDetector<ImageFloat32> detector = FactoryFiducial.pickAFiducial(...);
... additional fiducial specific configuration goes here ...
detector.setIntrinsic(param);
detector.detect(image);
Se3_F64 targetToSensor = new Se3_F64();
for (int i = 0; i < detector.totalFound(); i++){
	System.out.println("Target ID = "+detector.getId(i));
	System.out.println("Target width = "+detector.getWidth(i));
	detector.getFiducialToWorld(i,targetToSensor);
	System.out.println("Location:");
}

See the examples below for a more understanding of how to use these different types of fiducials.

Applets:

• Applet Fiducials

Examples:

• Square Binary Example
• Square Image Example
• Calibration Target Example