Tutorial Geometric Vision

The low level mathematics used to estimate the scene's structure and camera ego motion are contained in the Geometric Vision package in BoofCV. Most of the standard algorithms in this field are provided with numerious options for comptuing and refining constructs such as the Fundamental/Essential matrix, Trifocal Tensor, camera pose, and points/lines.

These algorithms are typically used in structure from motiom (SFM) and their correct usage is not trivial, see below for a list of books on the subject. The API is still being refined to help make this process easier. As is typical with BoofCV, most of the documentation on usage is provided in the form of examples and JavaDoc comments.

When reviewing the JavaDoc pay close attention to the type of inputs it takes (e.g. pixel or normalized image coordinates) and the direction of the reference frame transform. Pixels refers to coordinates in the image while normalized image coordinates are in Euclidean space and found by multiplying pixels by the inverse of the intrinsic camera calibration matirx.

To get started look at the following packages and classes:

• boofcv.abst.geo.*
• boofcv.alg.geo.*
• PerspectiveOps
• MultiViewOps
• FactoryMultiView
• FactoryTriangulate

Examples

• Compute Fundamental Matrix
• Multview Scene Reconstruction

Coordinate Systems

BoofCV follows standard practices in computer vision for it's coordinate systems. All 3D coordinate systems will be right handed.

Image Pixel Coordinates

The image coordinate system is the standard 2D one. Top left corner is (0,0). +x axis goes from image left to image right and +y axis from image top to image bottom.

Camera Coordinate System

Unless specified otherwise, the following camera coordinate system is used. The origin of the camera will be its optical center. Coordinate axises are defined as follows:

• +x axis is to the right
• +y axis is down
• +z axis is out of the camera

World Coordinates

The documentation frequently mentions world coordinates. This refers to the common coordinate system that you define. The only restriction is that it must be right handed. Specific applications inside of BoofCV might define a specific coordinate system, e.g. markers/fiducials. This should be defined in the JavaDoc, e.g. Square Binary Fiducial.

World Units

This just refers to the standard units used in your coordinate system. If you are using meters it's meters. If you don't care about the scale of something it doesn't matter how you define it.

Algorithm List

• Fundamental/Essential Matrix
  • Linear 8+ Points
  • Linear 7 Points
• Essential Matrix
  • Nister 5 Points
• Fundamental Matrix Optimization
  • Sampson Error
  • Epipolar Error
• Homography 4 Points (Linear)
• Homography Optimization
  • Sampson Error
  • Transfer Error
• Linear 6 Point Pose
• Linear Pixel Depth
• Perspective-N-Point (PnP)
  • Efficient PnP 4-Point (EPnP)
  • P3P Grunert
  • P3P Finsterwalder
• PnP Optimization
  • Euclidean Error
• Triangulation
  • Geometric
  • Linear
• Triangulation Optimization
  • Sampson Error
  • Euclidean Error
• Trifocal Tensor
  • Linear 7 point
• Decompose Essential
• Decompose Homography
• Bundle Adjustment
  • Dense (inefficient/slow)
• Stereo Rectification
  • Calibrated
  • Uncalibrated

Camera Model

ADD TO THIS

Recommend Reading

• Yi Ma, et. al., "An Invitation to 3-D Vision"
  • Algebraic approach to 3D vision.
  • Easy to use as a reference.
• R. Hartley, and A. Zisserman, "Multiple View Geometry in Computer Vision"
  • Geometric approach and the most popular book on this subject.
  • More content and good algorithms.
  • Mediocre index and tendency to not define terms in some chapters.