Tutorial Geometric Vision

The low level mathematics used to estimate the scene's structure and camera ego motion are contained in the Geometric Vision package in BoofCV. Most of the standard algorithms in this field are provided with numerious options for comptuing and refining constructs such as the Fundamental/Essential matrix, Trifocal Tensor, camera pose, and points/lines.

These algorithms are typically used in structure from motiom (SFM) and their correct usage is not trivial, see below for a list of books on the subject. The API is still being refined to help make this process easier. As is typical with BoofCV, most of the documentation on usage is provided in the form of examples and JavaDoc comments.

When reviewing the JavaDoc pay close attention to the type of inputs it takes (e.g. pixel or normalized image coordinates) and the direction of the reference frame transform. Pixels refers to coordinates in the image while normalized image coordinates are in Euclidean space and found by multiplying pixels by the inverse of the intrinsic camera calibration matirx.

To get started look at the following packages and classes:

• boofcv.abst.geo.*
• boofcv.alg.geo.*
• PerspectiveOps
• MultiViewOps
• FactoryMultiView
• FactoryTriangulate

Examples

• Compute Fundamental Matrix
• Multview Scene Reconstruction
• Full 3-View Uncalibrated to Metric Reconstruction

Videos

• Explanation of Camera Self Calibration

Coordinate Systems

See the Coordinate Systems page for a detailed description of all the coordinate systems used in BoofCV. For the most part BoofCV sticks with what is the closest to a standard in computer vision. Most of the time OpenCV and BoofCV use the same standards. If using a 3rd party library to calibrate a camera pay close attention on the section on camera spatial coordinates. There is no consensus and you might need to shift the image system by 0.5 a pixel. BoofCV and OpenCV use the same coordinate system, but for unknown reasons OpenCV calibration target detectors use the same coordinate system as Matlab and are shifted.

World Coordinates

The documentation frequently mentions world coordinates. This refers to the common coordinate system that you define. The only restriction is that it must be right handed. Specific applications inside of BoofCV might define a specific coordinate system, e.g. markers/fiducials. This should be defined in the JavaDoc, e.g. Square Binary Fiducial.

World Units

This just refers to the standard units used in your coordinate system. If you are using meters it's meters. If you don't care about the scale of something it doesn't matter how you define it.

Algorithm List

• Fundamental/Essential Matrix
  • Linear 8+ Points
  • Linear 7 Points
• Essential Matrix
  • Nister 5 Points
• Fundamental Matrix Optimization
  • Sampson Error
  • Epipolar Error
• Homography 4 Points (Linear)
• Homography Optimization
  • Sampson Error
  • Transfer Error
• Linear 6 Point Pose
• Linear Pixel Depth
• Perspective-N-Point (PnP)
  • Efficient PnP 4-Point (EPnP)
  • P3P Grunert
  • P3P Finsterwalder
• PnP Optimization
  • Euclidean Error
• Triangulation
  • Geometric
  • Linear
• Triangulation Optimization
  • Sampson Error
  • Euclidean Error
• Trifocal Tensor
  • Linear 7 point
• Decompose Essential
• Decompose Homography
• Sparse Bundle Adjustment
  • Metric 3D
  • Projective 3D
  • Projective Homogenous
• Stereo Rectification
  • Calibrated
  • Uncalibrated
• Self Calibration / Auto Calibration
  • Linear Dual Quadratic
  • Linear Pure Rotation
  • Estimate Plane at Infinity Given K
  • Refine Dual Quadratic
  • Guess and Check Focus ("Practical Autocalibration" 2010)

Camera Model

ADD TO THIS

Recommend Reading

• Yi Ma, et. al., "An Invitation to 3-D Vision"
  • Algebraic approach to 3D vision.
  • Easy to use as a reference.
• R. Hartley, and A. Zisserman, "Multiple View Geometry in Computer Vision"
  • Geometric approach and the most popular book on this subject.
  • More content and good algorithms.
  • Mediocre index and tendency to not define terms in some chapters.