3D Reconstruction on Desktop Tutorial - Revision history

Peter: /* Downloading */

2022-04-08T04:16:33Z

Downloading

← Older revision		Revision as of 04:16, 8 April 2022
Line 31:		Line 31:
	== Downloading ==		== Downloading ==

	[Applications\|Follow these instructions and download the pre-build applications jar]].		[[Applications\|Follow these instructions and download the pre-build applications jar]].

	== Example Data ==		== Example Data ==

Peter at 14:53, 19 October 2021

2021-10-19T14:53:52Z

← Older revision		Revision as of 14:53, 19 October 2021
Line 31:		Line 31:
	== Downloading ==		== Downloading ==

	~~Following [~~[Applications\|these instructions and download the pre-build applications jar]].		[Applications\|Follow these instructions and download the pre-build applications jar]].

	== Example Data ==		== Example Data ==
Line 51:		Line 51:
	ffmpeg -i your_video.mp4 images/frame%04d.png		ffmpeg -i your_video.mp4 images/frame%04d.png
	java -jar /path/to/boofcv/applications/applications.jar DownSelectVideoFramesFor3DApp -i images -o small -w 800 --MaxLength --MaxMotion 0.20 --MinMotion 0.05		java -jar /path/to/boofcv/applications/applications.jar DownSelectVideoFramesFor3DApp -i images -o small -w 800 --MaxLength --MaxMotion 0.20 --MinMotion 0.05
	java -jar /path/to/boofcv/applications/applications.jar ~~SceneReconstructionApp~~ -i small/ --Ordered -o reconstruction --ShowCloud --Verbose		java -jar /path/to/boofcv/applications/applications.jar SceneReconstruction -i small/ --Ordered -o reconstruction --ShowCloud --Verbose
	</pre>		</pre>

Peter: /* Building */

2021-10-19T04:04:24Z

Building

← Older revision		Revision as of 04:04, 19 October 2021
Line 29:		Line 29:
	These instructions are written for Ubuntu. You will need to have Java installed but everything else should take care of itself.		These instructions are written for Ubuntu. You will need to have Java installed but everything else should take care of itself.

	== ~~Building~~ ==		== Downloading ==

			Following [[Applications\|these instructions and download the pre-build applications jar]].
	~~You will need to latest and greatest code from Github~~ and build ~~the~~ applications jar.

	~~# [[Download#Bleeding_Edge \| Checkout the source code as described here]]~~
	~~# [[Manual#Building \| Build the code as described here.~~]]
	~~# Run this command: <pre>~~.~~/gradlew applicationsJar</pre>~~

	== Example Data ==		== Example Data ==

Peter: /* Building */

2021-10-19T03:19:47Z

Building

← Older revision		Revision as of 03:19, 19 October 2021
Line 30:		Line 30:

	== Building ==		== Building ==


	You will need to latest and greatest code from Github and build the applications jar.		You will need to latest and greatest code from Github and build the applications jar.

	<pre>		# [[Download#Bleeding_Edge \| Checkout the source code as described here]]
	~~git clone -b SNAPSHOT --recursive https://github.com/lessthanoptimal/BoofCV.git boofcv~~		# [[Manual#Building \| Build the code as described here.]]
	~~cd boofcv~~		# Run this command: <pre>./gradlew applicationsJar</pre>
	./gradlew applicationsJar
	</pre>

	== Example Data ==		== Example Data ==

Peter at 14:41, 26 July 2021

2021-07-26T14:41:33Z

← Older revision		Revision as of 14:41, 26 July 2021
Line 10:		Line 10:
	* Translational motion sideways from the object		* Translational motion sideways from the object
	* No calibration or hints required!		* No calibration or hints required!
			* Can split a video up into different objects automatically

	Bad		Bad
	* Dominant planes		* Drone footage is hit or miss
	* Drone ~~photo sets tend to fail more often than they work~~
	* Glare, smooth textureless objects, reflections, ... etc		* Glare, smooth textureless objects, reflections, ... etc
	* Does not use all the information available		* Does not use all the information available (EXIF, IMU, ... )
	* Can't link all the images into a single scene		* Can't link all the images into a single scene

Peter at 23:30, 6 June 2021

2021-06-06T23:30:36Z

← Older revision		Revision as of 23:30, 6 June 2021
Line 17:		Line 17:
	* Does not use all the information available		* Does not use all the information available
	* Can't link all the images into a single scene		* Can't link all the images into a single scene

			=== Informal Comparison to Meshroom/AliceVision ===
			By visual inspection it was clear that Meshroom does a better and more stable job in almost all scenarios for sparse reconstruction. The one exception was the 3-view case where BoofCV could often converge to a solution when Meshroom could not. While hard to compare, it does look like BoofCV's dense reconstruction might be better. Meshing needs to be added to BoofCV before a direct comparison can be done.

			BoofCV and AliceVision take two fairly different approaches to reconstruction, it should be interesting to see if the 3-view + self-calibration approach used in BoofCV can become as stable. AliceVision takes a more traditional approach where it guesses (using a nominal value) the focal length then expands the scene using 2-view pairs.



	== Requirements ==		== Requirements ==

Peter at 16:35, 4 June 2021

2021-06-04T16:35:23Z

← Older revision		Revision as of 16:35, 4 June 2021
Line 16:		Line 16:
	* Glare, smooth textureless objects, reflections, ... etc		* Glare, smooth textureless objects, reflections, ... etc
	* Does not use all the information available		* Does not use all the information available
			* Can't link all the images into a single scene

	== Requirements ==		== Requirements ==
Line 65:		Line 66:

	There are two key differences from the video example. There is no '--Ordered' flag which tells it the images are not in any particular order and '--TryHarder' flag. TryHarder isn't a requirement but it will increase the change of success significantly, but will run much slower. What this does is that it will consider more image features, reduce tolerances, and spend more time pruning outliers.		There are two key differences from the video example. There is no '--Ordered' flag which tells it the images are not in any particular order and '--TryHarder' flag. TryHarder isn't a requirement but it will increase the change of success significantly, but will run much slower. What this does is that it will consider more image features, reduce tolerances, and spend more time pruning outliers.

			== Viewing the Cloud ==

			The built in viewer gets a bit slow when you have millions of points but for the most part gets the job done. It uses WASD keys and the mouse to move around. Try using the mouse plus shift, cntrl, and alt keys and see what happens.

			<pre>
			java -jar /path/to/boofcv/applications/applications.jar --GUI
			</pre>

			Then click on the viewer button.

			Alternatively you can try 'osgviewer' but you might run into issues with it being too smart and auto scalling the scene. You will also want to turn off lighting (press 'l') so that colors are visible.

			<pre>
			osgviewer /path/to/reconstruction/saved_cloud.ply
			</pre>

Peter at 16:25, 4 June 2021

2021-06-04T16:25:11Z

← Older revision		Revision as of 16:25, 4 June 2021
Line 30:		Line 30:
	./gradlew applicationsJar		./gradlew applicationsJar
	</pre>		</pre>

			== Example Data ==

			<center>
			<gallery caption="Example Videos" heights=150 widths=200 >
			Image:Example_reconstruction_slippers.jpg\|Slippers Video\|link=https://boofcv.org/notwiki/reconstruction/slippers_01.mp4
			</gallery>
			</center>

			An attempt was made to not cherry pick examples too much. It should work on all of these if you follow the instrucitons below but it will not work perfectly. Most likely it will have issues linking all the images together and will use only a subset. If it does manage to link all the images the results might look worse since the code currently makes no attempt to filter out noisy points on the edges of objects. You get the idea.

	== Processing a Video ==		== Processing a Video ==

	First you will need a video to process. ~~Here are some~~ examples:		First you will need a video to process. Grab one of the examples above. Next you will need to 1) convert the video to a sequence of images, 2) downsample the images, 3) select key frames, 4) then perform the reconstruction. Here's how you can do all of that:
	* Video
	* Video

	Next you will need to convert the video to a sequence of images, ~~down sample~~ the images, select key frames, then perform the reconstruction. Here's how you can do all of that:

	<pre>		<pre>

Peter: Created page with "This tutorial is targeted at people who want to use BoofCV's 3D reconstruction pipeline, but are not developers and are not afraid to type commands in to the command line. 3D..."

2021-06-04T16:12:08Z

Created page with "This tutorial is targeted at people who want to use BoofCV's 3D reconstruction pipeline, but are not developers and are not afraid to type commands in to the command line. 3D..."

New page

This tutorial is targeted at people who want to use BoofCV's 3D reconstruction pipeline, but are not developers and are not afraid to type commands in to the command line. 3D Reconstruction / Photogrammetry in BoofCV will be a work in progress for a while. The fact that it works as well as it does is surprising.

== Setting Expectations ==

The initial development was focused on processing video sequences where it selects key frames to use. This is where it works best right now. However, most people like to take still photos and then create a 3D model from that. It's a bit hit or miss with sets of images. What I've noticed is that people don't have as much overlap between views and that makes things more difficult.

Good
* Video sequences with a mixture of close up and far away objects
* Highly textured scenes
* Translational motion sideways from the object
* No calibration or hints required!

Bad
* Dominant planes
* Drone photo sets tend to fail more often than they work
* Glare, smooth textureless objects, reflections, ... etc
* Does not use all the information available

== Requirements ==

These instructions are written for Ubuntu. You will need to have Java installed but everything else should take care of itself.

== Building ==

You will need to latest and greatest code from Github and build the applications jar.

<pre>
git clone -b SNAPSHOT --recursive https://github.com/lessthanoptimal/BoofCV.git boofcv
cd boofcv
./gradlew applicationsJar
</pre>

== Processing a Video ==

First you will need a video to process. Here are some examples:
* Video
* Video

Next you will need to convert the video to a sequence of images, down sample the images, select key frames, then perform the reconstruction. Here's how you can do all of that:

<pre>
mkdir images
ffmpeg -i your_video.mp4 images/frame%04d.png
java -jar /path/to/boofcv/applications/applications.jar DownSelectVideoFramesFor3DApp -i images -o small -w 800 --MaxLength --MaxMotion 0.20 --MinMotion 0.05
java -jar /path/to/boofcv/applications/applications.jar SceneReconstructionApp -i small/ --Ordered -o reconstruction --ShowCloud --Verbose
</pre>

If the reconstruction doesn't work very well you can try adding the '--TryHarder' flag, see below. If that doesn't work then see what happens when you process it as an unordered set of images.

The reconstruction (and debugging information) goes into the output directory 'reconstruction'. Most people are probably interested in the final cloud.ply document. There's a good chance that if you view it in a 3rd party application there will be scaling issues because BoofCV currently does not filter out points that are close to infinity...

== Processing Images ==

If you already have the set of images and you don't need to prune any of them then you can just invoke the command below:

<pre>
java -jar /path/to/boofcv/applications/applications.jar SceneReconstructionApp -i path/to/images/ --TryHarder -o reconstruction --ShowCloud --Verbose
</pre>

There are two key differences from the video example. There is no '--Ordered' flag which tells it the images are not in any particular order and '--TryHarder' flag. TryHarder isn't a requirement but it will increase the change of success significantly, but will run much slower. What this does is that it will consider more image features, reduce tolerances, and spend more time pruning outliers.