Difference between revisions of "3D Reconstruction on Desktop Tutorial"

From BoofCV
Jump to navigationJump to search
(Created page with "This tutorial is targeted at people who want to use BoofCV's 3D reconstruction pipeline, but are not developers and are not afraid to type commands in to the command line. 3D...")
 
m
Line 30: Line 30:
./gradlew applicationsJar
./gradlew applicationsJar
</pre>
</pre>
== Example Data ==
<center>
<gallery caption="Example Videos" heights=150 widths=200 >
Image:Example_reconstruction_slippers.jpg|Slippers Video|link=https://boofcv.org/notwiki/reconstruction/slippers_01.mp4
</gallery>
</center>
An attempt was made to not cherry pick examples too much. It should work on all of these if you follow the instrucitons below but it will not work perfectly. Most likely it will have issues linking all the images together and will use only a subset. If it does manage to link all the images the results might look worse since the code currently makes no attempt to filter out noisy points on the edges of objects. You get the idea.


== Processing a Video ==
== Processing a Video ==


First you will need a video to process. Here are some examples:
First you will need a video to process. Grab one of the examples above. Next you will need to 1) convert the video to a sequence of images, 2) downsample the images, 3) select key frames, 4) then perform the reconstruction. Here's how you can do all of that:
* Video
* Video
 
Next you will need to convert the video to a sequence of images, down sample the images, select key frames, then perform the reconstruction. Here's how you can do all of that:


<pre>
<pre>

Revision as of 09:25, 4 June 2021

This tutorial is targeted at people who want to use BoofCV's 3D reconstruction pipeline, but are not developers and are not afraid to type commands in to the command line. 3D Reconstruction / Photogrammetry in BoofCV will be a work in progress for a while. The fact that it works as well as it does is surprising.

Setting Expectations

The initial development was focused on processing video sequences where it selects key frames to use. This is where it works best right now. However, most people like to take still photos and then create a 3D model from that. It's a bit hit or miss with sets of images. What I've noticed is that people don't have as much overlap between views and that makes things more difficult.

Good

  • Video sequences with a mixture of close up and far away objects
  • Highly textured scenes
  • Translational motion sideways from the object
  • No calibration or hints required!

Bad

  • Dominant planes
  • Drone photo sets tend to fail more often than they work
  • Glare, smooth textureless objects, reflections, ... etc
  • Does not use all the information available

Requirements

These instructions are written for Ubuntu. You will need to have Java installed but everything else should take care of itself.

Building

You will need to latest and greatest code from Github and build the applications jar.

git clone -b SNAPSHOT --recursive https://github.com/lessthanoptimal/BoofCV.git boofcv
cd boofcv
./gradlew applicationsJar

Example Data

An attempt was made to not cherry pick examples too much. It should work on all of these if you follow the instrucitons below but it will not work perfectly. Most likely it will have issues linking all the images together and will use only a subset. If it does manage to link all the images the results might look worse since the code currently makes no attempt to filter out noisy points on the edges of objects. You get the idea.

Processing a Video

First you will need a video to process. Grab one of the examples above. Next you will need to 1) convert the video to a sequence of images, 2) downsample the images, 3) select key frames, 4) then perform the reconstruction. Here's how you can do all of that:

mkdir images
ffmpeg -i your_video.mp4 images/frame%04d.png
java -jar /path/to/boofcv/applications/applications.jar DownSelectVideoFramesFor3DApp -i images -o small -w 800 --MaxLength --MaxMotion 0.20 --MinMotion 0.05
java -jar /path/to/boofcv/applications/applications.jar SceneReconstructionApp -i small/ --Ordered -o reconstruction --ShowCloud --Verbose

If the reconstruction doesn't work very well you can try adding the '--TryHarder' flag, see below. If that doesn't work then see what happens when you process it as an unordered set of images.

The reconstruction (and debugging information) goes into the output directory 'reconstruction'. Most people are probably interested in the final cloud.ply document. There's a good chance that if you view it in a 3rd party application there will be scaling issues because BoofCV currently does not filter out points that are close to infinity...

Processing Images

If you already have the set of images and you don't need to prune any of them then you can just invoke the command below:

java -jar /path/to/boofcv/applications/applications.jar SceneReconstructionApp -i path/to/images/ --TryHarder -o reconstruction --ShowCloud --Verbose

There are two key differences from the video example. There is no '--Ordered' flag which tells it the images are not in any particular order and '--TryHarder' flag. TryHarder isn't a requirement but it will increase the change of success significantly, but will run much slower. What this does is that it will consider more image features, reduce tolerances, and spend more time pruning outliers.