Difference between revisions of "Tutorial Geometric Vision"

From BoofCV
Jump to navigationJump to search
m
m
 
(19 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Geometric Computer Vision Tutorial =
The low level mathematics used to estimate the scene's structure and camera ego motion are contained in the Geometric Vision package in BoofCV.  Most of the standard algorithms in this field are provided with numerious options for comptuing and refining constructs such as the Fundamental/Essential matrix, Trifocal Tensor, camera pose, and points/lines.


A major component of 3D computer vision is the study of [http://en.wikipedia.org/wiki/Epipolar_geometry epipolar geometry], the geometry of two views.  BoofCV provides many standard algorithms for mathematically describing features across two views and more.  Below is a list of these algorithms.  The API is still being worked on and a more detailed tutorial will be written later on.
These algorithms are typically used in structure from motiom (SFM) and their correct usage is not trivial, see below for a list of books on the subject.  The API is still being refined to help make this process easier.  As is typical with BoofCV, most of the documentation on usage is provided in the form of examples and JavaDoc comments.


Algorithm List
When reviewing the JavaDoc pay close attention to the type of inputs it takes (e.g. pixel or normalized image coordinates) and the direction of the reference frame transform.  Pixels refers to coordinates in the image while normalized image coordinates are in Euclidean space and found by multiplying pixels by the inverse of the intrinsic camera calibration matirx.
* Fundamental Matrix 8 Points (Linear)
 
* Fundamental Matrix 7 Points (Linear)
To get started look at the following packages and classes:
* Fundamental Optimization
* boofcv.abst.geo.*
* boofcv.alg.geo.*
* PerspectiveOps
* MultiViewOps
* FactoryMultiView
* FactoryTriangulate
 
== Examples ==
 
* [[Example_Fundamental_Matrix| Compute Fundamental Matrix]]
* [[Example_Structure_from_Motion| Multview Scene Reconstruction]]
* [[Example_Three_View_Stereo_Uncalibrated| Full 3-View Uncalibrated to Metric Reconstruction]]
 
== Videos ==
* [https://www.youtube.com/watch?v=O_yAdsT8d84 Explanation of Camera Self Calibration]
 
== Coordinate Systems ==
 
See the [[Coordinate_Systems|Coordinate Systems]] page for a detailed description of all the coordinate systems used in BoofCV. For the most part BoofCV sticks with what is the closest to a standard in computer vision. Most of the time OpenCV and BoofCV use the same standards. If using a 3rd party library to calibrate a camera pay close attention on the section on camera spatial coordinates. There is no consensus and you might need to shift the image system by 0.5 a pixel. BoofCV and OpenCV use the same coordinate system, but for unknown reasons OpenCV calibration target detectors use the same coordinate system as Matlab and are shifted.
 
 
=== World Coordinates ===
The documentation frequently mentions world coordinates. This refers to the common coordinate system that you define. The only restriction is that it must be right handed. Specific applications inside of BoofCV might define a specific coordinate system, e.g. markers/fiducials. This should be defined in the JavaDoc, e.g. [http://boofcv.org/javadoc/boofcv/alg/fiducial/square/DetectFiducialSquareBinary.html Square Binary Fiducial].
 
== World Units ==
 
This just refers to the standard units used in your coordinate system. If you are using meters it's meters. If you don't care about the scale of something it doesn't matter how you define it.
 
== Algorithm List ==
 
* Fundamental/Essential Matrix  
** Linear 8+ Points
** Linear 7 Points
* Essential Matrix  
** Nister 5 Points
* Fundamental Matrix Optimization
** Sampson Error
** Sampson Error
** Epipolar Error
** Epipolar Error
Line 13: Line 48:
** Sampson Error
** Sampson Error
** Transfer Error
** Transfer Error
* Efficient PnP 4-Point
* Linear 6 Point Pose  
* Linear 6 Point Pose  
* Linear Pixel Depth
* Linear Pixel Depth
* Triangulation Geometric
* Perspective-N-Point (PnP)
* Triangulation Linear
** Efficient PnP 4-Point (EPnP)
** P3P Grunert
** P3P Finsterwalder
* PnP Optimization
** Euclidean Error
* Triangulation  
** Geometric
** Linear
* Triangulation Optimization
* Triangulation Optimization
** Sampson Error
** Sampson Error
** Euclidean Error
** Euclidean Error
* Trifocal Tensor
** Linear 7 point
* Decompose Essential
* Decompose Essential
* Decompose Homography
* Decompose Homography
* Sparse Bundle Adjustment
** Metric 3D
** Projective 3D
** Projective Homogenous
* Stereo Rectification
** Calibrated
** Uncalibrated
* Self Calibration / Auto Calibration
** Linear Dual Quadratic
** Linear Pure Rotation
** Estimate Plane at Infinity Given K
** Refine Dual Quadratic
** Guess and Check Focus ("Practical Autocalibration" 2010)
== Camera Model ==
ADD TO THIS


Examples
= Recommend Reading =
* [[Example_Fundamental_Matrix| Compute Fundamental Matrix]]


Related
* [http://www.amazon.com/gp/product/0387008934/ref=as_li_ss_tl?ie=UTF8&tag=boofcv-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0387008934 Yi Ma, et. al., "An Invitation to 3-D Vision"]
* Zhang 99 Camera Calibration
** Algebraic approach to 3D vision.
* Various different feature trackers and detectors
** Easy to use as a reference.
* [http://www.amazon.com/gp/product/0521540518/ref=as_li_ss_tl?ie=UTF8&tag=boofcv-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0521540518 R. Hartley, and A. Zisserman, "Multiple View Geometry in Computer Vision"]
** Geometric approach and the most popular book on this subject. 
** More content and good algorithms. 
** Mediocre index and tendency to not define terms in some chapters.

Latest revision as of 20:20, 24 January 2020

The low level mathematics used to estimate the scene's structure and camera ego motion are contained in the Geometric Vision package in BoofCV. Most of the standard algorithms in this field are provided with numerious options for comptuing and refining constructs such as the Fundamental/Essential matrix, Trifocal Tensor, camera pose, and points/lines.

These algorithms are typically used in structure from motiom (SFM) and their correct usage is not trivial, see below for a list of books on the subject. The API is still being refined to help make this process easier. As is typical with BoofCV, most of the documentation on usage is provided in the form of examples and JavaDoc comments.

When reviewing the JavaDoc pay close attention to the type of inputs it takes (e.g. pixel or normalized image coordinates) and the direction of the reference frame transform. Pixels refers to coordinates in the image while normalized image coordinates are in Euclidean space and found by multiplying pixels by the inverse of the intrinsic camera calibration matirx.

To get started look at the following packages and classes:

  • boofcv.abst.geo.*
  • boofcv.alg.geo.*
  • PerspectiveOps
  • MultiViewOps
  • FactoryMultiView
  • FactoryTriangulate

Examples

Videos

Coordinate Systems

See the Coordinate Systems page for a detailed description of all the coordinate systems used in BoofCV. For the most part BoofCV sticks with what is the closest to a standard in computer vision. Most of the time OpenCV and BoofCV use the same standards. If using a 3rd party library to calibrate a camera pay close attention on the section on camera spatial coordinates. There is no consensus and you might need to shift the image system by 0.5 a pixel. BoofCV and OpenCV use the same coordinate system, but for unknown reasons OpenCV calibration target detectors use the same coordinate system as Matlab and are shifted.


World Coordinates

The documentation frequently mentions world coordinates. This refers to the common coordinate system that you define. The only restriction is that it must be right handed. Specific applications inside of BoofCV might define a specific coordinate system, e.g. markers/fiducials. This should be defined in the JavaDoc, e.g. Square Binary Fiducial.

World Units

This just refers to the standard units used in your coordinate system. If you are using meters it's meters. If you don't care about the scale of something it doesn't matter how you define it.

Algorithm List

  • Fundamental/Essential Matrix
    • Linear 8+ Points
    • Linear 7 Points
  • Essential Matrix
    • Nister 5 Points
  • Fundamental Matrix Optimization
    • Sampson Error
    • Epipolar Error
  • Homography 4 Points (Linear)
  • Homography Optimization
    • Sampson Error
    • Transfer Error
  • Linear 6 Point Pose
  • Linear Pixel Depth
  • Perspective-N-Point (PnP)
    • Efficient PnP 4-Point (EPnP)
    • P3P Grunert
    • P3P Finsterwalder
  • PnP Optimization
    • Euclidean Error
  • Triangulation
    • Geometric
    • Linear
  • Triangulation Optimization
    • Sampson Error
    • Euclidean Error
  • Trifocal Tensor
    • Linear 7 point
  • Decompose Essential
  • Decompose Homography
  • Sparse Bundle Adjustment
    • Metric 3D
    • Projective 3D
    • Projective Homogenous
  • Stereo Rectification
    • Calibrated
    • Uncalibrated
  • Self Calibration / Auto Calibration
    • Linear Dual Quadratic
    • Linear Pure Rotation
    • Estimate Plane at Infinity Given K
    • Refine Dual Quadratic
    • Guess and Check Focus ("Practical Autocalibration" 2010)

Camera Model

ADD TO THIS

Recommend Reading