Difference between revisions of "Tutorial Fiducials"

From BoofCV
Jump to navigationJump to search
m
m
Line 31: Line 31:
</center>
</center>


== Quick Start ==
= Quick Start =


# Calibrate your camera and save results ([[Tutorial_Camera_Calibration|Tutorial]])
# Calibrate your camera and save results ([[Tutorial_Camera_Calibration|Tutorial]])
# Print binary fiducial (see below creating/printing)
#* Technically optional, but highly recommended
# Print binary fiducial, e.g. [http://boofcv.org/notwiki/fiducials/square0643.eps Binary #0643]
# Launch fiducial webcam application
# Launch fiducial webcam application
# Point camera at fiducial
# Point camera at fiducial


For the last step you need to launch TrackFiducialWebcam in boofcv/application.  The easiest way to do that is with the following Gradle script.
== Launching Fiducial Application ==
 
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
gradle trackFiducial -Pcamera=0 -Pintrinsic="/path/to/intrinsic.xml"
cd boofcv/applications
gradle applciationsJar
java -cp applications.jar boofcv.app.WebcamTrackFiducial --Resolution=640:480 BINARY
</syntaxhighlight>
</syntaxhighlight>
"camera" is used to specify which camera and "intrinsic" the intrinsic calibration.  If you haven't calibrated your camera yet, but still want to see something you can omit the "intrinsic" parameter and it will guess the parameters.  The results will not be as good, even if there is very little lens distortion, but you can see something.
 
If you calibrated your camera you can do the following and get better results:
 
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
gradle trackFiducial -Pcamera=0
java -cp applications.jar boofcv.app.WebcamTrackFiducial --Intrinsic=intrinsic.xml BINARY
</syntaxhighlight>
</syntaxhighlight>


 
To get a list of commands and see how to track other types of fiducials just enter the command with no arguments.
== High Level Interface ==
<syntaxhighlight lang="bash">
 
java -cp applications.jar boofcv.app.WebcamTrackFiducial
FiducialDetector is an easy to use high-level interface for fiducials.  FactoryFiducial is the easiest way to create instances of different fiducial types and it hides much of the complexity.  Some detectors require additional information after construction. For example, square image fiducials require images be provided for each target it can detect.  A sketch of how to process a single image is shown below.
<syntaxhighlight lang="java">
FiducialDetector<ImageFloat32> detector = FactoryFiducial.pickAFiducial(...);
... additional fiducial specific configuration goes here ...
detector.setIntrinsic(param);
detector.detect(image);
Se3_F64 targetToSensor = new Se3_F64();
for (int i = 0; i < detector.totalFound(); i++){
System.out.println("Target ID = "+detector.getId(i));
detector.getFiducialToWorld(i,targetToSensor);
System.out.println("Location:");
}
</syntaxhighlight>
</syntaxhighlight>


Applets:
== Printable Fiducials ==
* [[Applet_Fiducials| Applet Fiducials]]


Examples:
The following is a printable documents for all the types of fiducials supported in BoofCV.  Print these to get started quickly, but creating your own is also easy.
* [[Example Fiducial Square Binary| Square Binary Example]]
* [[Example Fiducial Square Image| Square Image Example]]
* [[Example_Calibration_Target_Pose|Calibration Target Example]]


== Square Fiducials ==
* Square Binary
*# [http://boofcv.org/notwiki/fiducials/square0643.eps File Binary #0643]
*# [http://boofcv.org/notwiki/fiducials/square0284.eps File Binary #0284]
* Square Image
*# [http://boofcv.org/notwiki/fiducials/dog.eps File Dog]
*# [http://boofcv.org/notwiki/fiducials/text.eps File Text]
* Calibration Chessboard
*# [http://boofcv.org/notwiki/calibration/letter_chess.ps Letter Sized Paper: Chessboard, 5 by 7, 30mm Squares]
*# [http://boofcv.org/notwiki/calibration/A4_chess.ps A4 Sized Paper: Chessboard, 5 by 8, 30mm Squares]
*# [http://boofcv.org/notwiki/calibration/A1_chess.ps A1 Sized Paper: Chessboard, 8 by 12, 60mm Squares]
* Calibration Square Grid
*# [http://boofcv.org/notwiki/calibration/letter_square.ps Letter Sized Paper: Square Grid, 5 by 7, 30mm Squares]
*# [http://boofcv.org/notwiki/calibration/A4_square.ps A4 Sized Paper: Square Grid, 5 by 9, 30mm Squares]
*# [http://boofcv.org/notwiki/calibration/A1_square.ps A1 Sized Paper: Square Grid, 7 by 11 60mm Squares]


[[File:Square fiducial parts.png|thumb|300px|A) White outside region makes it easier to detect. B) Black square border which.  C) Encoded image or pattern.]]
= Creating your Own Fiducial =


All square fiducials share a common code base.  A target contains a black square of constant width and inside there is an image or pattern. The pattern is used to uniquely identify the fiducial and determine its orientation.  A full 6-DOF pose is estimated from these fiducials.  These targets are inspired by ARToolkit, but the code is not a port and was developed from scratched to fully utilize existing code in BoofCV.
For square fiducials, a convenient command-line application is provided which can create printable EPS documents which contain one or more fiducials on themFor calibration targets, prefabricated patterns are provided which can also be printed.


When detecting a square fiducial, the first step is to threshold the image. FactoryFiducial provides "robust" and "fast" techniques and the only difference between them is if they use an adaptive technique or a fixed threshold, respectivelyLocally adaptive thresholding is invariant to local changes in lighting.  The next step is to find the contour of blobs in the image. Clearly invalid contours are pruned and a polygon fit to the contour.  This contour is used to provide the initial estimate of the squares edges.  An expectation-maximization algorithm is used to fit lines to the contour and the corners are found by the intersection of the lines.  Once the corners are found a homography is computed and then decomposed to return the pose. 
== Square Binary ==
Fiducials can be made using the applications.jar you created earlierUsual for complete instructions just enter in the classes name with no arguments.


Once the pose is known, perspective distortion can be removed and a synthetic image created. Orientation ambiguity is resolved using the fiducials pattern inside the square. For the binary pattern 4 corners are used. For image based fiducials, 4 different possible orientations are considered and the best match used.
<syntaxhighlight lang="bash">
java -cp applications.jar boofcv.app.CreateFiducialSquareBinaryEPS -OutputFile=fiducial.eps -PrintInfo -Units=cm 12 284
</syntaxhighlight>


The specifics for each type of square fiducial is discussed below.
That will create a printable fiducial.eps file that encodes the number 284 in a square that's 12 centimeters.


=== Square Binary ===
== Square Image ==


The square binary fiducial encodes a 12-bit number, 4096 possible values, using a binary pattern. The number is encoded by breaking up the inner portion into 16 squares in a 4x4 grid.  Three of the corners are always white and one black.  This is how it resolves an orientation ambiguity.   
A fiducial can be easily created from any image using "applications.jar".   


A new fiducial can be created using the ''DetectFiducialSquareBinary'' application.  For easy of use a Gradle script has been provided:
<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
gradle fiducialBinary -Pwidth=10 -Pnumber=325
java -cp applications.jar boofcv.app.CreateFiducialSquareImageEPS -OutputFile=fiducial.eps -PrintInfo -Units=cm 12 pentarose.png
 
:applications:classes UP-TO-DATE
:applications:fiducialBinary
Target width 10.0 (cm)  number = 325
101000101000
 
BUILD SUCCESSFUL
</syntaxhighlight>
</syntaxhighlight>
This will create a pattern which is 10cm wide and encodes the number 325.  The output will be saved in "boofcv/applications/pattern.eps" file.  See the top figure the resulting pattern.
This will create a pattern which is 10cm wide and encodes the image contained in 'dog.png'.  The output will be saved in "boofcv/applications/fiducial_image.eps" file.  See the top figure the resulting pattern.


Detection is easy enough using the high level Fiducial interfaceCode example and sample targets below.
= How Do Square Fiducials Work? =
[[File:Square fiducial parts.png|thumb|300px|A) White outside region makes it easier to detect. B) Black square border whichC) Encoded image or pattern.]]


* [[Example Fiducial Square Binary| Binary Detection Code Example]]
All square fiducials share a common code base. A target contains a black square of constant width and inside there is an image or pattern. The pattern is used to uniquely identify the fiducial and determine its orientation. A full 6-DOF pose is estimated from these fiducials. These targets are inspired by ARToolkit, but the code is not a port and was developed from scratched. 
* [http://boofcv.org/notwiki/fiducials/square0643.eps File Binary #0643]
* [http://boofcv.org/notwiki/fiducials/square0284.eps File Binary #0284]


=== Square Image ===
When detecting a square fiducial, the first step is to threshold the image.  FactoryFiducial provides "robust" and "fast" techniques and the only difference between them is if they use an adaptive technique or a fixed threshold, respectively.  Locally adaptive thresholding is invariant to local changes in lighting.  The next step is to find the contour of blobs in the image.  Clearly invalid contours are pruned and a polygon fit to the contour.  This contour is used to provide the initial estimate of the squares edges.  An expectation-maximization algorithm is used to fit lines to the contour and the corners are found by the intersection of the lines.  Once the corners are found a homography is computed and then decomposed to return the pose. 


Square image fiducials identify a target by embedding an image inside a square. The theoretical maximum number of unique fiducials is quite large, but in practice is limited by camera resolution and processing powerThe time to process an image increases linearly O(N) with the number images.  For a small number of images constant time overhead (binaryization and contour identification) will dominate because images are encoded efficiently as binary numbers in integers and fast bitwise operators used to compare.
Once the pose is known, perspective distortion can be removed and a synthetic image created.  Orientation ambiguity is resolved using the fiducials pattern inside the square.  For the binary pattern 4 corners are used.  For image based fiducials, 4 different possible orientations are considered and the best match used.


A fiducial can be created from any image.  CreateFiducialSquareImageEPS is used to create new postscript fiducial files and will automatically rescale the image so that it is square and ensure that it's the correct size.  For easy of use a Gradle script has been provided:
= Programming =
<syntaxhighlight lang="bash">
gradle fiducialImage -Pwidth=10.0 -Pimage="../data/applet/fiducial/image/dog.png"


:applications:classes UP-TO-DATE
All of these different types of fiducials can be used through a high level interface, *FiducialDetector*FactoryFiducial is the easiest way to create instances of different fiducial types and it hides much of the complexity.  Some detectors require additional information after construction. For example, square image fiducials require images be provided for each target it can detect.  A sketch of how to process a single image is shown below.
:applications:fiducialImage
Target width 10.0 (cm) image = dog.png


BUILD SUCCESSFUL
<syntaxhighlight lang="java">
FiducialDetector<ImageFloat32> detector = FactoryFiducial.pickAFiducial(...);
... additional fiducial specific configuration goes here ...
detector.setIntrinsic(param);
detector.detect(image);
Se3_F64 targetToSensor = new Se3_F64();
for (int i = 0; i < detector.totalFound(); i++){
System.out.println("Target ID = "+detector.getId(i));
System.out.println("Target width = "+detector.getWidth(i));
detector.getFiducialToWorld(i,targetToSensor);
System.out.println("Location:");
}
</syntaxhighlight>
</syntaxhighlight>
This will create a pattern which is 10cm wide and encodes the image contained in 'dog.png'.  The output will be saved in "boofcv/applications/fiducial_image.eps" file.  See the top figure the resulting pattern.


* [[Example Fiducial Square Image| Image Detection Code Example]]
See the examples below for a more understanding of how to use these different types of fiducials.
* [http://boofcv.org/notwiki/fiducials/dog.eps File Dog]
* [http://boofcv.org/notwiki/fiducials/text.eps File Text]


== Calibration Target ==
Applets:
* [[Applet_Fiducials| Applet Fiducials]]


The patterns used to calibrate the camera are also fiducials.  These are designed for high accuracy when close to the camera and tolerant to moderate lens distortion.  While quite good up close their accuracy degrades the farther away and the more acute the viewing angle is, perhaps faster than other target types.
Examples:
 
* [[Example Fiducial Square Binary| Square Binary Example]]
There are two significant draw backs to using calibration targets as pose estimation fiducials.  1) Most common configurations only give a partial pose estimate. 2) Only one can be visible at a time.  A partial pose estimate is given because of symmetry along each axis.  It is possible to choose a number of squares such that some of the symmetry goes away in one or more axis, but still requires you to be careful.  When being developed it was assumed that only one is visible at a time and there is no ID encoded into the pattern.  Thus if there is more than one there will be confusion.
* [[Example Fiducial Square Image| Square Image Example]]
 
* [[Example_Calibration_Target_Pose|Calibration Target Example]]
A set of patterns created for calibration can be found in 'boofcv/data/evaluation/calibration' or downloaded below:
* [http://boofcv.org/notwiki/calibration/letter_chess.ps Letter Sized Paper: Chessboard, 5 by 7, 30mm Squares]
* [http://boofcv.org/notwiki/calibration/letter_square.ps Letter Sized Paper: Square Grid, 5 by 7, 30mm Squares]
* [http://boofcv.org/notwiki/calibration/A4_chess.ps A4 Sized Paper: Chessboard, 5 by 8, 30mm Squares]
* [http://boofcv.org/notwiki/calibration/A4_square.ps A4 Sized Paper: Square Grid, 5 by 9, 30mm Squares]
* [http://boofcv.org/notwiki/calibration/A1_chess.ps A1 Sized Paper: Chessboard, 8 by 12, 60mm Squares]
* [http://boofcv.org/notwiki/calibration/A1_square.ps A1 Sized Paper: Square Grid, 7 by 11 60mm Squares]
where A1, A4, and letter refers to the paper size.  Chess for chessboard pattern and square for square grid.  See [[Tutorial_Camera_Calibration|calibration tutorial]] for more information.
 
* [[Example_Calibration_Target_Pose| Calibration Target Example]]

Revision as of 20:28, 20 September 2015


In computer vision, a fiducial marker is a known object which can be identified and have its pose estimated. BoofCV provides built in support several different fiducials, which can be easily printed. Applications are provided for automatically creating printable postscript files and a high level interface for detection, identication and pose estimation.

There are two types of fiducials supported in BoofCV, square and calibration targets. Square fiducials encode a pattern inside a black square box. These targets can be uniquely identified and provide a full pose estimate. Calibration targets fiducials are repurposed targets used to calibrate cameras. Calibration fiducials tend to provide very accurate pose estimation when close to the camera, but can have difficulty as they move away. There are two significant disadvantage for calibration targets. 1) They don't provide a unique ID. 2) Most patterns are not fully orientation invariant. You can see the lack of rotation invariance when it suddenly flips 180 degrees.

Fiducial Summary Table

Type Variant Speed (FPS) Unique Pose Accuracy
Square Binary Fast 175 4096 Full Good
Robust 67
Square Image Fast 170 Full Good
Robust 66
Calibration Chessboard 69 1 Partial Best Close

Speed to detect multiple fiducials in a 640x480 image on a Intel Core i7-2600 3.4 Ghz. Unique max number of unique targets it can identify. Pose indicates if a full 6-DOF estimate is found or subset. Infinity symbol really means "lots".

Quick Start

  1. Calibrate your camera and save results (Tutorial)
    • Technically optional, but highly recommended
  2. Print binary fiducial, e.g. Binary #0643
  3. Launch fiducial webcam application
  4. Point camera at fiducial

Launching Fiducial Application

cd boofcv/applications
gradle applciationsJar
java -cp applications.jar boofcv.app.WebcamTrackFiducial --Resolution=640:480 BINARY

If you calibrated your camera you can do the following and get better results:

java -cp applications.jar boofcv.app.WebcamTrackFiducial --Intrinsic=intrinsic.xml BINARY

To get a list of commands and see how to track other types of fiducials just enter the command with no arguments.

java -cp applications.jar boofcv.app.WebcamTrackFiducial

Printable Fiducials

The following is a printable documents for all the types of fiducials supported in BoofCV. Print these to get started quickly, but creating your own is also easy.

Creating your Own Fiducial

For square fiducials, a convenient command-line application is provided which can create printable EPS documents which contain one or more fiducials on them. For calibration targets, prefabricated patterns are provided which can also be printed.

Square Binary

Fiducials can be made using the applications.jar you created earlier. Usual for complete instructions just enter in the classes name with no arguments.

java -cp applications.jar boofcv.app.CreateFiducialSquareBinaryEPS -OutputFile=fiducial.eps -PrintInfo -Units=cm 12 284

That will create a printable fiducial.eps file that encodes the number 284 in a square that's 12 centimeters.

Square Image

A fiducial can be easily created from any image using "applications.jar".

java -cp applications.jar boofcv.app.CreateFiducialSquareImageEPS -OutputFile=fiducial.eps -PrintInfo -Units=cm 12 pentarose.png

This will create a pattern which is 10cm wide and encodes the image contained in 'dog.png'. The output will be saved in "boofcv/applications/fiducial_image.eps" file. See the top figure the resulting pattern.

How Do Square Fiducials Work?

A) White outside region makes it easier to detect. B) Black square border which. C) Encoded image or pattern.

All square fiducials share a common code base. A target contains a black square of constant width and inside there is an image or pattern. The pattern is used to uniquely identify the fiducial and determine its orientation. A full 6-DOF pose is estimated from these fiducials. These targets are inspired by ARToolkit, but the code is not a port and was developed from scratched.

When detecting a square fiducial, the first step is to threshold the image. FactoryFiducial provides "robust" and "fast" techniques and the only difference between them is if they use an adaptive technique or a fixed threshold, respectively. Locally adaptive thresholding is invariant to local changes in lighting. The next step is to find the contour of blobs in the image. Clearly invalid contours are pruned and a polygon fit to the contour. This contour is used to provide the initial estimate of the squares edges. An expectation-maximization algorithm is used to fit lines to the contour and the corners are found by the intersection of the lines. Once the corners are found a homography is computed and then decomposed to return the pose.

Once the pose is known, perspective distortion can be removed and a synthetic image created. Orientation ambiguity is resolved using the fiducials pattern inside the square. For the binary pattern 4 corners are used. For image based fiducials, 4 different possible orientations are considered and the best match used.

Programming

All of these different types of fiducials can be used through a high level interface, *FiducialDetector*. FactoryFiducial is the easiest way to create instances of different fiducial types and it hides much of the complexity. Some detectors require additional information after construction. For example, square image fiducials require images be provided for each target it can detect. A sketch of how to process a single image is shown below.

FiducialDetector<ImageFloat32> detector = FactoryFiducial.pickAFiducial(...);
... additional fiducial specific configuration goes here ...
detector.setIntrinsic(param);
detector.detect(image);
Se3_F64 targetToSensor = new Se3_F64();
for (int i = 0; i < detector.totalFound(); i++){
	System.out.println("Target ID = "+detector.getId(i));
	System.out.println("Target width = "+detector.getWidth(i));
	detector.getFiducialToWorld(i,targetToSensor);
	System.out.println("Location:");
}

See the examples below for a more understanding of how to use these different types of fiducials.

Applets:

Examples: