Difference between revisions of "Tutorial Fiducials"

From BoofCV
Jump to navigationJump to search
m
m
 
(30 intermediate revisions by 2 users not shown)
Line 3: Line 3:
Image:Fiducial_squre_binary.png| Square Binary
Image:Fiducial_squre_binary.png| Square Binary
Image:Fiducial_square_image.png| Square Image
Image:Fiducial_square_image.png| Square Image
Image:Calib_target_chess_small.png| Calibration Target
File:Calibration_letter_chessboard_7x5.png| Calibration Target
Image:Fiducial square binary detected.jpg| Cubes rendered on top of detected fiducials
Image:Fiducial square binary detected.jpg| Cubes rendered on top of detected fiducials
</gallery>
</gallery>
</center>
</center>


In computer vision, a fiducial marker is a known object which can be easily identified.  BoofCV provides built in support several different fiducials, all of which can be easily printed on paper.  Applications are provided for creating your own postscript files.  BoofCV library provides a single high level interface for detection, identification and pose estimation of all fiducials. Alternatively you can use the low level API and access fiducial specific information. 


In computer vision, a fiducial marker is a known object from which can be identified and its pose estimated. BoofCV provides built in support several different fiducials which can be easily printed.  Applications are provided for automatically creating postscript files for the printer and a high level interface for detecting, identifying and pose estimation. 
<center>[https://youtu.be/qJWDK_FrgHE Fiducial Video Overview]</center>


There are two types of fiducials supported in BoofCV, square and calibration targets.  Square fiducials encode a pattern inside a black square box.  These targets can be uniquely identified and provide a pose estimate.  Calibration targets fiducials are repurposed targets used to calibrate cameras.  Calibration fiducials tend to provide very accurate pose estimation when close to the camera, but can have difficulty as they move away.  There are two significant disadvantage for calibration targets. 1) They don't provide a unique ID.  2) Most patterns are not fully orientation invariant.  You can see the lack of rotation invariance when it suddenly flips 180 degrees.
There are two types of fiducials supported in BoofCV, square and calibration targets.  Square fiducials encode a large number of unique patterns inside a black square box.  These targets can be uniquely identified and provide a full pose estimate.  Calibration targets fiducials are repurposed targets used to calibrate cameras.  Calibration fiducials tend to provide very accurate pose estimation when close to the camera, but can have difficulty as they move away.  There are two significant disadvantage for calibration targets. 1) They don't provide a unique ID.  2) Most patterns are not fully orientation invariant.  You can see the lack of rotation invariance when it suddenly flips 180 degrees.


<center>
<center>
'''Fiducial Summary Table'''
'''Fiducial Summary Table'''
{| class="wikitable"
{| class="wikitable"
! Type        !! Variant !! Speed (FPS)  !! Unique    !! Pose    !! Accuracy
! Type        !! Variant     !! Speed (FPS)  !! Unique    !! Pose    !! Accuracy
|-
|-
|Square Binary|| Fast   || 175          || 4096      || Full    || Good
|Square Binary|| Fast       ||style="text-align:center;"| 175          || Varies    || Full    || Good
|-
|-
|            || Robust || 67          ||          ||        ||
|            || Robust     ||style="text-align:center;"| 67          ||          ||        ||
|-
|-
|Square Image || Fast   || 170          || &infin;  || Full    || Good
|Square Image || Fast       ||style="text-align:center;"| 170          || &infin;  || Full    || Good
|-
|-
|            || Robust || 66          ||          ||        ||
|            || Robust     ||style="text-align:center;"| 66          ||          ||        ||
|-
|-
|Calibration  ||        ||X             || 1        || Partial || Best Close
|Calibration  || Chessboard  ||style="text-align:center;"| 97          || 1         || Partial || Best Close
|-
|             || Square Grid ||style="text-align:center;"| 100          || 1        || Partial || Best Close
|}
|}
Speed to detect multiple fiducials in a 640x480 image on a Intel Core i7-2600 3.4 Ghz.
''Speed'' to detect multiple fiducials in a 640x480 image on a Intel Core i7-2600 3.4 Ghz.  ''Unique'' max number of unique targets it can identify.  ''Pose'' indicates if a full 6-DOF estimate is found or subset.  Infinity symbol really means "lots".  Binary fiducials can be configured with different inner grids.  Total number of unique fiducials will very from 32 to 1.15e18 for a 3x3 or 8x8 grid, respectively.
</center>
</center>


{| class="wikitable"
= Quick Start =
! Orange
! Apple
| style="text-align:right;"| 12,333.00
|-
| Bread
| Pie
| style="text-align:right;"| 500.00
|- style="font-style: italic; color: green;"
| Butter
| Ice cream
| style="text-align:right;"| 1.00
|}


== Quick Start ==
# Calibrate your camera and save results ([[Tutorial_Camera_Calibration|Tutorial]])
 
#* Technically optional, but highly recommended
# Calibrate your camera and save results ([[Tutorial_Camera_Calibration|Tutorial])
# Print binary fiducial, e.g. [http://boofcv.org/notwiki/fiducials/square0643.eps Binary #0643]
# Print binary fiducial (see below creating/printing)
# Launch fiducial webcam application
# Launch fiducial webcam application
# Point camera at fiducial
# Point camera at fiducial


For the last step you need to launch TrackFiducialWebcam in boofcv/applicationThe easiest way to do that is with the following Gradle script.
== Application ==
 
To help you get started and quickly test ideas there's a fiducial application included with BoofCV.  It will detect fiducials, then draw a 1/2 cube on top of them so you can see how accurate its 3D estimate isTo run the application do the following:
 
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
gradle trackFiducial -Pcamera=0 -Pintrinsic="/path/to/intrinsic.xml"
cd boofcv/applications
gradle applicationsJar
java -jar applications.jar WebcamTrackFiducial --Resolution=640:480 BINARY
</syntaxhighlight>
</syntaxhighlight>
"camera" is used to specify which camera and "intrinsic" the intrinsic calibration.  If you haven't calibrated your camera yet, but still want to see something you can omit the "intrinsic" parameter and it will guess the parameters.  The results will not be as good, even if there is very little lens distortion, but you can see something.
 
If you calibrated your camera you can do the following and get better results:
 
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
gradle trackFiducial -Pcamera=0
java -jar applications.jar WebcamTrackFiducial --Intrinsic=intrinsic.xml BINARY
</syntaxhighlight>
</syntaxhighlight>


To get a list of commands and see how to track other types of fiducials just enter the command with no arguments.
<syntaxhighlight lang="bash">
java -jar applications.jar WebcamTrackFiducial
</syntaxhighlight>
== Printable Fiducials ==
The following is a printable documents for all the types of fiducials supported in BoofCV.  Print these to get started quickly, but creating your own is also easy.
{| class="wikitable"
! Square Binary !!  Chessboard
|-
|
* [http://boofcv.org/notwiki/fiducials/square0643.eps File 4x4 Binary #0643]
* [http://boofcv.org/notwiki/fiducials/square0284.eps File 4x4 Binary #0284]
|
* [http://boofcv.org/notwiki/calibration/letter_chess.ps Letter Sized Paper: Chessboard, 5 by 7, 30mm Squares]
* [http://boofcv.org/notwiki/calibration/A4_chess.ps A4 Sized Paper: Chessboard, 5 by 8, 30mm Squares]
* [http://boofcv.org/notwiki/calibration/A1_chess.ps A1 Sized Paper: Chessboard, 8 by 12, 60mm Squares]
|-
! Square Image !! Square grid
|-
|
* [http://boofcv.org/notwiki/fiducials/10cm_pentarose.eps Pentarose]
* [http://boofcv.org/notwiki/fiducials/10cm_yinyang.eps Yin Yang]
* [http://boofcv.org/notwiki/fiducials/all_a4.eps Sheet of All, A4]
* [http://boofcv.org/notwiki/fiducials/all_letter.eps Sheet of All, Letter]
* [https://github.com/lessthanoptimal/BoofCV-Data/tree/master/applet/fiducial/image/eps bunch more]
|
* [http://boofcv.org/notwiki/calibration/letter_square.ps Letter Sized Paper: Square Grid, 3 by 4, 30mm Squares]
* [http://boofcv.org/notwiki/calibration/A4_square.ps A4 Sized Paper: Square Grid, 3 by 5, 30mm Squares]
* [http://boofcv.org/notwiki/calibration/A1_square.ps A1 Sized Paper: Square Grid, 4 by 6 60mm Squares]
|}
= Creating your Own Fiducial =
For square fiducials, a convenient command-line application is provided which can create printable postscript (.ps) documents which contain one or more fiducials on them.  For calibration targets, prefabricated patterns are provided which can also be printed.


== High Level Interface ==
== Square Binary ==
Fiducials can be made using the applications.jar you created earlier.  Usual for complete instructions just enter in the classes name with no arguments.


FiducialDetector is an easy to use high-level interface for fiducials.  FactoryFiducial is the easiest way to create instances of different fiducial types and it hides much of the complexity.  Some detectors require additional information after construction. For example, square image fiducials require images be provided for each target it can detect.  A sketch of how to process a single image is shown below.
<syntaxhighlight lang="bash">
<syntaxhighlight lang="java">
java -jar applications.jar CreateFiducialSquareBinary -OutputFile=fiducial.ps -PrintInfo -Units=cm 12 284
FiducialDetector<ImageFloat32> detector = FactoryFiducial.pickAFiducial(...);
... additional fiducial specific configuration goes here ...
detector.setIntrinsic(param);
detector.detect(image);
Se3_F64 targetToSensor = new Se3_F64();
for (int i = 0; i < detector.totalFound(); i++){
System.out.println("Target ID = "+detector.getId(i));
detector.getFiducialToWorld(i,targetToSensor);
System.out.println("Location:");
}
</syntaxhighlight>
</syntaxhighlight>


Examples:
That will create a printable fiducial.ps file that encodes the number 284 in a 4x4 square that's 12 centimeters.  The number of elements which compose its inner grid and the width of the outside border are both configurable.  By default all code assumes a 4x4 grid and 0.25 fractional border width.  This produces a reasonable compose between detection robustness and the number of features it can encode, which is 4096.
* [[Example Fiducial Square Binary| Square Binary Example]]
* [[Example Fiducial Square Image| Square Image Example]]
* [[Example Fiducial Calibration Target| Calibration Target Example]]


== Square Fiducials ==
== Square Image ==


[[File:Square fiducial parts.png|thumb|300px|A) White outside region makes it easier to detect. B) Black square border whichC) Encoded image or pattern.]]
A fiducial can be easily created from any image using "applications.jar".   


All square fiducials share a common code base. A target contains a black square of constant width and inside there is an image or pattern. The pattern is used to uniquely identify the fiducial and determine its orientation. A full 6-DOF pose is estimated from these fiducialsThese targets are inspired by ARToolkit, but the code is not a port and was developed from scratched to fully utilize existing code in BoofCV.
<syntaxhighlight lang="bash">
java -jar applications.jar CreateFiducialSquareImage -OutputFile=fiducial.ps -PrintInfo -Units=cm 12 pentarose.png
</syntaxhighlight>
This will create a pattern which is 12cm wide and encodes the image contained in 'pentarose.png'The output will be saved in "fiducial.ps" file.


The initial processing step is to threshold the image. BoofCV provides a various thresholding techniques for doing so.  FactoryFiducial provides "robust" and "fast" techniques.  Robust will use a locally adaptive algorithm which is invariant to local changes in lighting while fast uses a constant threshold. The next step is to find the contour of blobs in the image.  Clearly invalid contours are pruned and a polygon fit to the contour.  This contour is used to provide the initial estimate of the squares edges.  An expectation-maximumization algorithm is used to fit lines to the contour and the corners are found by the intersection of the lines.  Once the corners are found a homography is computed and then decomposed to return the pose. 
Not feeling very creative or just want to see some example images?  Several patterns are contained in [https://github.com/lessthanoptimal/BoofCV-Data/tree/master/example/fiducial/image/patterns "data/example/fiducial/image/patterns/"].


One the pose is known perspective distortion can be used to remove and a synthetic image created.  The fiducial is uniquely identified using the synthetic image.  Orientation ambiguity is resolved using the fiducials pattern inside the square.  For the binary pattern 4 corners are used.  For the image 4 different possible orientations are considered and the best match used.
= How Do Square Fiducials Work? =
[[File:Square fiducial parts.png|thumb|300px|The origin of the fiducial is at its center with x and y axises as shown.  The fiducial itself can be divided into three regions, a) white outside region which provides contrast against the black border, b) black square border, and c) encoded image or pattern.]]


The specifics for each type of square fiducial is discussed below.
All square fiducials share a common code base.  A target contains a black square of constant width and inside there is an image or pattern. The pattern is used to uniquely identify the fiducial and determine its orientation.  A full 6-DOF pose is estimated from these fiducials.  These targets are inspired by ARToolkit, but the code is not a port and was developed from scratched.


=== Square Binary ===
'''Processing Steps'''
# Create binary image by thresholding
# Contours of dark objects
# Contour to crude quadrilateral
# Sub-pixel refinement of quadrilateral
# Undistort quadrilateral image and match to known patterns
# Extract 3D pose of fiducial


The square binary fiducial encodes a 12-bit number, 4096 possible values, using a binary pattern. The number is encoded by breaking up the inner portion into 16 squares in a 4x4 grid.  Three of the corners are always white and one black.  This is how it resolves an orientation ambiguity. 


A new fiducial can be created using the ''DetectFiducialSquareBinary'' application. For easy of use a Gradle script has been provided:
''Step 1:'' [[Example_Thresholding|Thresholding]] is performed using either a user configurable fixed threshold or locally adaptive threshold, fast and robust respectively. While slower robust can handle variations in lighting much better.
<syntaxhighlight lang="bash">
gradle fiducialBinary -Pwidth=10 -Pnumber=325


:applications:classes UP-TO-DATE
''Step 2:'' Contours are found from the binary blobs using [http://boofcv.org/javadoc/boofcv/alg/filter/binary/LinearContourLabelChang2004.html Chang 2004].
:applications:fiducialBinary
Target width 10.0 (cm)  number = 325
101000101000


BUILD SUCCESSFUL
''Step 3:'' The contour is converted into a polygon using a [[Example_Fit_Polygon|split and merge]] techniqueOnly four sided polygons are accepted.
</syntaxhighlight>
This will create a pattern which is 10cm wide and encodes the number 325.  The output will be saved in "boofcv/applications/pattern.eps" fileSee the top figure the resulting pattern.


Detection is easy enough using the high level Fiducial interfaceSee the example below for the details.
''Step 4:'' The crude polygons are refined to subpixel accuracy.  This is done by iteratively refining the estimate of each side's lineThat is done by computing the difference of line integrals tangential to each line, then using their intensity values to compute a weighted line.


[[Example Fiducial Square Binary| Binary Detection Example]]
''Step 5:'' The four corners are now known to a high level of precision, these are then used to compute a homography which is then used to generate an undistorted square image of the inner pattern. This pattern is then used by the specific algorithm to identify a known target or compute a number.  Noisy images are rejected here.
   
=== Square Image ===


Square image fiducials identify a target by embedding an image inside a square. The theoretical maximum number of unique fiducials is quite large, but in practice is limited by camera resolution and processing power. The time to process an image increases linearly O(N) with the number imagesFor a small number of images constant time overhead (binaryization and contour identification) will dominate because images are encoded efficiently as binary numbers in integers and fast bitwise operators used to compare.   
''Step 6:'' The pose is computed through a combination of [[Example_PnP|P3P and P4P]] techniques. The one with the smallest residual error is selected.  Then non-linear refinement is performed. Care is taken to reduce the effects of head on small fiducials, which have a degenerate geometryIn that situation significant changes in orientation result in only a small change on the imageThat is handled by placing a greater emphasis on reprojection error caused by orientation.


A fiducial can be created from any image.  CreateFiducialSquareImageEPS is used to create new postscript fiducial files and will automatically rescale the image so that it is square and ensure that it's the correct size.  For easy of use a Gradle script has been provided:
NOTE: Steps 2 to 4 are carried out by the [[Example_Detect_Black_Polygons|black polygon detector]].
<syntaxhighlight lang="bash">
gradle fiducialImage -Pwidth=10.0 -Pimage="../data/applet/fiducial/image/dog.png"


:applications:classes UP-TO-DATE
== Square Binary ==
:applications:fiducialImage
Target width 10.0 (cm)  image = dog.png


BUILD SUCCESSFUL
The square binary fiducial encodes a 12-bit number, 4096 possible values, using a binary pattern. The number is encoded by breaking up the inner portion into 16 squares in a 4x4 grid. Three of the corners are always white and one black. This is how it resolves an orientation ambiguity.  
</syntaxhighlight>
This will create a pattern which is 10cm wide and encodes the image contained in 'dog.png'. The output will be saved in "boofcv/applications/fiducial_image.eps" file.  See the top figure the resulting pattern.


== Calibration Target ==
== Square Image ==


The patterns used to calibrate the camera are also fiducialsThese are designed for high accuracy when close to the camera and tolerant to moderate lens distortion. While quite good up close their accuracy degrades the farther away and the more acute the viewing angle is, perhaps faster than other target types.
When an image is loaded into this type of fiducial it is first converted into a square image then down sampled into a low resolution image and encoded efficiently When processing a video feed and a fiducial is detected the pattern is undistorted as usual. Then the hamming distance between the just observed pattern and all the known patterns is found.  The pattern with the best score within tolerance is accepted.


There are two significant draw backs to using calibration targets as pose estimation fiducials.  1) They only give a partial pose estimate. 2) Only one can be visible at a time.  A partial pose estimate is given because of symmetry along each axis.  It is possible to choose a number of squares such that some of the symmetry goes away in one or more axis, but still requires you to be careful.  When being developed it was assumed that only one is visible at a time and there is no ID encoded into the pattern.  Thus if there is more than one there will be confusion.
= Programming =


A set of patterns created for calibration can be found in 'boofcv/data/evaluation/calibration'
All of these different types of fiducials can be used through a high level interface, *FiducialDetector*FactoryFiducial is the easiest way to create instances of different fiducial types and it hides much of the complexitySome detectors require additional information after construction. For example, square image fiducials require images be provided for each target it can detectA sketch of how to process a single image is shown below.
<pre>
A1_chess.ps  A4_chess.ps  letter_chess.ps
A1_square.ps A4_square.ps letter_square.ps
</pre>
where A1, A4, and letter refers to the paper size.  Chess for chessboard pattern and square for square gridSee [[Tutorial_Camera_Calibration|calibration tutorial]] for more information.


* [[Example Fiducial Calibration Target| Calibration Target Example]]
<syntaxhighlight lang="java">
FiducialDetector<GrayF32> detector = FactoryFiducial.pickAFiducial(...);
... additional fiducial specific configuration goes here ...
detector.setIntrinsic(param);
detector.detect(image);
Se3_F64 targetToSensor = new Se3_F64();
for (int i = 0; i < detector.totalFound(); i++){
System.out.println("Target ID = "+detector.getId(i));
System.out.println("Target width = "+detector.getWidth(i));
detector.getFiducialToCamera(i,targetToSensor);
System.out.println("Location:");
}
</syntaxhighlight>
 
See the examples below for a more understanding of how to use these different types of fiducials.
 
Examples:
* [[Example Fiducial Square Binary| Square Binary Example]]
* [[Example Fiducial Square Image| Square Image Example]]
* [[Example_Calibration_Target_Pose|Calibration Target Example]]

Latest revision as of 10:50, 20 January 2018

In computer vision, a fiducial marker is a known object which can be easily identified. BoofCV provides built in support several different fiducials, all of which can be easily printed on paper. Applications are provided for creating your own postscript files. BoofCV library provides a single high level interface for detection, identification and pose estimation of all fiducials. Alternatively you can use the low level API and access fiducial specific information.

Fiducial Video Overview

There are two types of fiducials supported in BoofCV, square and calibration targets. Square fiducials encode a large number of unique patterns inside a black square box. These targets can be uniquely identified and provide a full pose estimate. Calibration targets fiducials are repurposed targets used to calibrate cameras. Calibration fiducials tend to provide very accurate pose estimation when close to the camera, but can have difficulty as they move away. There are two significant disadvantage for calibration targets. 1) They don't provide a unique ID. 2) Most patterns are not fully orientation invariant. You can see the lack of rotation invariance when it suddenly flips 180 degrees.

Fiducial Summary Table

Type Variant Speed (FPS) Unique Pose Accuracy
Square Binary Fast 175 Varies Full Good
Robust 67
Square Image Fast 170 Full Good
Robust 66
Calibration Chessboard 97 1 Partial Best Close
Square Grid 100 1 Partial Best Close

Speed to detect multiple fiducials in a 640x480 image on a Intel Core i7-2600 3.4 Ghz. Unique max number of unique targets it can identify. Pose indicates if a full 6-DOF estimate is found or subset. Infinity symbol really means "lots". Binary fiducials can be configured with different inner grids. Total number of unique fiducials will very from 32 to 1.15e18 for a 3x3 or 8x8 grid, respectively.

Quick Start

  1. Calibrate your camera and save results (Tutorial)
    • Technically optional, but highly recommended
  2. Print binary fiducial, e.g. Binary #0643
  3. Launch fiducial webcam application
  4. Point camera at fiducial

Application

To help you get started and quickly test ideas there's a fiducial application included with BoofCV. It will detect fiducials, then draw a 1/2 cube on top of them so you can see how accurate its 3D estimate is. To run the application do the following:

cd boofcv/applications
gradle applicationsJar
java -jar applications.jar WebcamTrackFiducial --Resolution=640:480 BINARY

If you calibrated your camera you can do the following and get better results:

java -jar applications.jar WebcamTrackFiducial --Intrinsic=intrinsic.xml BINARY

To get a list of commands and see how to track other types of fiducials just enter the command with no arguments.

java -jar applications.jar WebcamTrackFiducial

Printable Fiducials

The following is a printable documents for all the types of fiducials supported in BoofCV. Print these to get started quickly, but creating your own is also easy.

Square Binary Chessboard
Square Image Square grid

Creating your Own Fiducial

For square fiducials, a convenient command-line application is provided which can create printable postscript (.ps) documents which contain one or more fiducials on them. For calibration targets, prefabricated patterns are provided which can also be printed.

Square Binary

Fiducials can be made using the applications.jar you created earlier. Usual for complete instructions just enter in the classes name with no arguments.

java -jar applications.jar CreateFiducialSquareBinary -OutputFile=fiducial.ps -PrintInfo -Units=cm 12 284

That will create a printable fiducial.ps file that encodes the number 284 in a 4x4 square that's 12 centimeters. The number of elements which compose its inner grid and the width of the outside border are both configurable. By default all code assumes a 4x4 grid and 0.25 fractional border width. This produces a reasonable compose between detection robustness and the number of features it can encode, which is 4096.

Square Image

A fiducial can be easily created from any image using "applications.jar".

java -jar applications.jar CreateFiducialSquareImage -OutputFile=fiducial.ps -PrintInfo -Units=cm 12 pentarose.png

This will create a pattern which is 12cm wide and encodes the image contained in 'pentarose.png'. The output will be saved in "fiducial.ps" file.

Not feeling very creative or just want to see some example images? Several patterns are contained in "data/example/fiducial/image/patterns/".

How Do Square Fiducials Work?

The origin of the fiducial is at its center with x and y axises as shown. The fiducial itself can be divided into three regions, a) white outside region which provides contrast against the black border, b) black square border, and c) encoded image or pattern.

All square fiducials share a common code base. A target contains a black square of constant width and inside there is an image or pattern. The pattern is used to uniquely identify the fiducial and determine its orientation. A full 6-DOF pose is estimated from these fiducials. These targets are inspired by ARToolkit, but the code is not a port and was developed from scratched.

Processing Steps

  1. Create binary image by thresholding
  2. Contours of dark objects
  3. Contour to crude quadrilateral
  4. Sub-pixel refinement of quadrilateral
  5. Undistort quadrilateral image and match to known patterns
  6. Extract 3D pose of fiducial


Step 1: Thresholding is performed using either a user configurable fixed threshold or locally adaptive threshold, fast and robust respectively. While slower robust can handle variations in lighting much better.

Step 2: Contours are found from the binary blobs using Chang 2004.

Step 3: The contour is converted into a polygon using a split and merge technique. Only four sided polygons are accepted.

Step 4: The crude polygons are refined to subpixel accuracy. This is done by iteratively refining the estimate of each side's line. That is done by computing the difference of line integrals tangential to each line, then using their intensity values to compute a weighted line.

Step 5: The four corners are now known to a high level of precision, these are then used to compute a homography which is then used to generate an undistorted square image of the inner pattern. This pattern is then used by the specific algorithm to identify a known target or compute a number. Noisy images are rejected here.

Step 6: The pose is computed through a combination of P3P and P4P techniques. The one with the smallest residual error is selected. Then non-linear refinement is performed. Care is taken to reduce the effects of head on small fiducials, which have a degenerate geometry. In that situation significant changes in orientation result in only a small change on the image. That is handled by placing a greater emphasis on reprojection error caused by orientation.

NOTE: Steps 2 to 4 are carried out by the black polygon detector.

Square Binary

The square binary fiducial encodes a 12-bit number, 4096 possible values, using a binary pattern. The number is encoded by breaking up the inner portion into 16 squares in a 4x4 grid. Three of the corners are always white and one black. This is how it resolves an orientation ambiguity.

Square Image

When an image is loaded into this type of fiducial it is first converted into a square image then down sampled into a low resolution image and encoded efficiently . When processing a video feed and a fiducial is detected the pattern is undistorted as usual. Then the hamming distance between the just observed pattern and all the known patterns is found. The pattern with the best score within tolerance is accepted.

Programming

All of these different types of fiducials can be used through a high level interface, *FiducialDetector*. FactoryFiducial is the easiest way to create instances of different fiducial types and it hides much of the complexity. Some detectors require additional information after construction. For example, square image fiducials require images be provided for each target it can detect. A sketch of how to process a single image is shown below.

FiducialDetector<GrayF32> detector = FactoryFiducial.pickAFiducial(...);
... additional fiducial specific configuration goes here ...
detector.setIntrinsic(param);
detector.detect(image);
Se3_F64 targetToSensor = new Se3_F64();
for (int i = 0; i < detector.totalFound(); i++){
	System.out.println("Target ID = "+detector.getId(i));
	System.out.println("Target width = "+detector.getWidth(i));
	detector.getFiducialToCamera(i,targetToSensor);
	System.out.println("Location:");
}

See the examples below for a more understanding of how to use these different types of fiducials.

Examples: