3D Reconstruction on Desktop Tutorial

This tutorial is targeted at people who want to use BoofCV's 3D reconstruction pipeline, but are not developers and are not afraid to type commands in to the command line. 3D Reconstruction / Photogrammetry in BoofCV will be a work in progress for a while. The fact that it works as well as it does is surprising.

Setting Expectations

The initial development was focused on processing video sequences where it selects key frames to use. This is where it works best right now. However, most people like to take still photos and then create a 3D model from that. It's a bit hit or miss with sets of images. What I've noticed is that people don't have as much overlap between views and that makes things more difficult.

Good

• Video sequences with a mixture of close up and far away objects
• Highly textured scenes
• Translational motion sideways from the object
• No calibration or hints required!

Bad

• Dominant planes
• Drone photo sets tend to fail more often than they work
• Glare, smooth textureless objects, reflections, ... etc
• Does not use all the information available

Requirements

These instructions are written for Ubuntu. You will need to have Java installed but everything else should take care of itself.

Building

You will need to latest and greatest code from Github and build the applications jar.

git clone -b SNAPSHOT --recursive https://github.com/lessthanoptimal/BoofCV.git boofcv
cd boofcv
./gradlew applicationsJar

Processing a Video

First you will need a video to process. Here are some examples:

• Video
• Video

Next you will need to convert the video to a sequence of images, down sample the images, select key frames, then perform the reconstruction. Here's how you can do all of that:

mkdir images
ffmpeg -i your_video.mp4 images/frame%04d.png
java -jar /path/to/boofcv/applications/applications.jar DownSelectVideoFramesFor3DApp -i images -o small -w 800 --MaxLength --MaxMotion 0.20 --MinMotion 0.05
java -jar /path/to/boofcv/applications/applications.jar SceneReconstructionApp -i small/ --Ordered -o reconstruction --ShowCloud --Verbose

If the reconstruction doesn't work very well you can try adding the '--TryHarder' flag, see below. If that doesn't work then see what happens when you process it as an unordered set of images.

The reconstruction (and debugging information) goes into the output directory 'reconstruction'. Most people are probably interested in the final cloud.ply document. There's a good chance that if you view it in a 3rd party application there will be scaling issues because BoofCV currently does not filter out points that are close to infinity...

Processing Images

If you already have the set of images and you don't need to prune any of them then you can just invoke the command below:

java -jar /path/to/boofcv/applications/applications.jar SceneReconstructionApp -i path/to/images/ --TryHarder -o reconstruction --ShowCloud --Verbose

There are two key differences from the video example. There is no '--Ordered' flag which tells it the images are not in any particular order and '--TryHarder' flag. TryHarder isn't a requirement but it will increase the change of success significantly, but will run much slower. What this does is that it will consider more image features, reduce tolerances, and spend more time pruning outliers.