Difference between revisions of "Tutorial Videos and Webcams"

From BoofCV
Jump to navigationJump to search
m
 
(11 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Videos and Webcams in BoofCV =
BoofCV focuses on image processing and computer vision algorithms and for the most part relies on other libraries for input and output.  For example, BoofCV does not define GUI widgets but uses Swing instead.  The same philosophy applies to capturing images from videos and webcams.  Unfortunately the built in support in Java for reading videos and webcams is very limited.  Fortunately there are several 3rd party libraries that do provide support for reading in videos and webcams!


BoofCV focuses on image processing and computer vision algorithms and for the most part relies on other libraries for input and output.  For example, BoofCV does not define image widgets but uses Swing instead.  The same philosophy applies to capturing images from videos and webcams.  Unfortunately the built in support that Java provides for reading videos and webcams is very limited.
= Reading Videos =
This section describes different approaches that can be used to read in video files.


The limited built in support that BoofCV provides itself for reading videos directly comes in the form of MJPEG videos and image sequence.  MJPEG is a very simple video format that encapsulates a sequence of video images.  MJPEG is not the most efficient format, but it is very easy to read and write.  Its inclusion into BoofCV is primarily to provide video support to applets.  An image sequence contained in a directly can also be read as if it was a video.
== Loading Image Files ==


Third party libraries are required for reading webcams or other video formatsSupport for these third party libraries can be found in the 'boofcv/integration' directoryCurrently wrappers are only provided for XugglerLibraries which are not 'supported' can still be used easily with BoofCV, you just need to provide the routines to convert the images.
The most straight forward way to read video is to convert it into an image sequenceHow you do that is dependent on what OS your using and is left as an exercise for the readerTypically the way images are handled on the desktop is to first read them in as a BufferedImage and then convert that image into a BoofCV image formatHere's an example for where a sequence of images is loaded from a directory.


== Converting to MJPEG ==
<syntaxhighlight lang="java">
List<BufferedImage> images = UtilImageIO.loadImages("path/to/images","jpg");
GrayU8 gray = new GrayU8(1,1);
for( BufferedImage image : images ) {
    ConvertBufferedImage.convertFrom(image,gray,true);
}
</syntaxhighlight>


To read a video and not mess with third party libraries the easiest way to do so is convert the video to MJPEG.  The easiest way in Linux to create an MJPEG from some other video format is using ffmpegBelow is a command line example. 
Individual images are also loaded easily using UtilImageIO.
 
 
== Reading MJPEG ==
 
The only video format which BoofCV natively supports is MJPEG and image sequence.  MJPEG is a very simple video format that is for the most part a series of JPEG images concatenated together.  MJPEG's inclusion in BoofCV was primarily to provide video support in Applets.  To read a video and not mess with third party libraries your options are to convert it into an image sequence (previous section) or into an MJPEG.  In Linux ffmpeg can be used to convert videos and image sequences into MJPEG.  An example is shown below:
<pre>
<pre>
ffmpeg -i FILENAME -sameq example.mjpeg
ffmpeg -i FILENAME -sameq example.mjpeg
Line 15: Line 27:
Other flags can be added to resize the image or adjust image quality.  There are a few videos where ffmpeg will produce an MJPEG which BoofCV cannot read, but this is not common.
Other flags can be added to resize the image or adjust image quality.  There are a few videos where ffmpeg will produce an MJPEG which BoofCV cannot read, but this is not common.


== Xuggler Integration ==
== Other Video Formats ==
The easiest way to use Video in BoofCV is to use the default MediaManager.  It checks your class path to see what you have installed and will use the best option available.  SimpleImageSequence is an interface that provides an easy to use and common interface for different image streams.  Make sure you add the BoofCV module [https://github.com/lessthanoptimal/BoofCV/tree/master/integration/javacv JavaCV] to your project!  JavaCV provides a wrapper around [https://ffmpeg.org/ FFMPEG].  If you don't include this package you will be stuck with just MJPEG files.
 
<syntaxhighlight lang="java">
MediaManager media = DefaultMediaManager.INSTANCE;
SimpleImageSequence<Planar<GrayU8>> video = media.openVideo(fileName, ImageType.pl(3,GrayU8.class);


Xuggler support is still being worked on to make the API easier to deal with.  To read a video file as a SimpleImageSequence use XugglerSimplified. For example:
while( video.hasNext() ) {
<pre>
  Planar<GrayU8> input = video.next();
SimpleImageSequence<ImageFloat32> videoLeft =
  ... process ...
  new XugglerSimplified<ImageFloat32>("FileName.avi", ImageFloat32.class);
}
</pre>
</syntaxhighlight>


Be sure to correctly setup all the classpaths for Xugger and to include the BoofCVXuggler.jar in your classpath. BoofCVXuggler.jar can be downloaded online or compiled yourself using ant scripts provided in 'boofcv/integration/xuggler'.
The following is a list of alternatives.  Note that a built in MediaManager is not provided for these options.
* [http://www.xuggle.com/xuggler/ Xuggler] used supported but after a long protracted death is no longer available
* Java Media Framework (JMF)
** http://www.oracle.com/technetwork/java/javase/tech/index-jsp-140239.html
** Provided by Oracle and supports a few formats.  Known for being old and poorly supported.


See also integration JavaDoc
= Webcams =
* [http://boofcv.org/javadoc/integration/ JavaDoc]


= Other Options =
The recommended way to use webcams with BoofCV is with [http://webcam-capture.sarxos.pl/ Webcam Capture].  Webcam Capture is very easy to use and works out of the box on Linux, Windows, and MacOS.  The BoofCV integration package only provides a single function for opening up a camera at a resolution as close to the one you requested as possible.  You need to be sure that the BoofCV [https://github.com/lessthanoptimal/BoofCV/tree/master/integration/WebcamCapture WebcamCapture] module is included with your project.


The easiest way to integrate a 3rd party library with BoofCV is to read in a frame as a BufferedImage then convert it into a native BoofCV image format using ConvertBufferedImage.  An alternative is to write a wrapper that implements SimpleImageSequence, which is used by several examples.
[[Example Webcam Capture|Webcam Capture Example]]


'''Video Libraries'''
Other Options:
* [http://code.google.com/p/v4l4j/ Video for Linux for Java (V4L4J)] used to be supported and can still be used but you will need to write your own code to extract the images.


* Java Media Framework (JMF)
= BufferedImage =
** http://www.oracle.com/technetwork/java/javase/tech/index-jsp-140239.html
** Provided by Oracle and supports a few formats.  Known for being old and poorly supported.


'''Webcams'''
As previously mentioned BufferedImages can be converted into a BoofCV image using ConvertBufferedImage.  Thus any library which can read a file into BufferedImages can be used with BoofCV.


* Video for Linux for Java (V4L4J)
= Android =
** http://code.google.com/p/v4l4j/
** I have had good success with it in the past


If you know of some other options please post a message to BoofCV's message board (see sidebar link) letting us know about it!  Especially if you have suggestions for Windows and MacOS.
See [[Android_support|Android Support]] for how to convert Android images into a BoofCV image.

Latest revision as of 18:41, 3 January 2017

BoofCV focuses on image processing and computer vision algorithms and for the most part relies on other libraries for input and output. For example, BoofCV does not define GUI widgets but uses Swing instead. The same philosophy applies to capturing images from videos and webcams. Unfortunately the built in support in Java for reading videos and webcams is very limited. Fortunately there are several 3rd party libraries that do provide support for reading in videos and webcams!

Reading Videos

This section describes different approaches that can be used to read in video files.

Loading Image Files

The most straight forward way to read video is to convert it into an image sequence. How you do that is dependent on what OS your using and is left as an exercise for the reader. Typically the way images are handled on the desktop is to first read them in as a BufferedImage and then convert that image into a BoofCV image format. Here's an example for where a sequence of images is loaded from a directory.

List<BufferedImage> images = UtilImageIO.loadImages("path/to/images","jpg");
GrayU8 gray = new GrayU8(1,1);
for( BufferedImage image : images ) {
    ConvertBufferedImage.convertFrom(image,gray,true);
}

Individual images are also loaded easily using UtilImageIO.


Reading MJPEG

The only video format which BoofCV natively supports is MJPEG and image sequence. MJPEG is a very simple video format that is for the most part a series of JPEG images concatenated together. MJPEG's inclusion in BoofCV was primarily to provide video support in Applets. To read a video and not mess with third party libraries your options are to convert it into an image sequence (previous section) or into an MJPEG. In Linux ffmpeg can be used to convert videos and image sequences into MJPEG. An example is shown below:

ffmpeg -i FILENAME -sameq example.mjpeg

Other flags can be added to resize the image or adjust image quality. There are a few videos where ffmpeg will produce an MJPEG which BoofCV cannot read, but this is not common.

Other Video Formats

The easiest way to use Video in BoofCV is to use the default MediaManager. It checks your class path to see what you have installed and will use the best option available. SimpleImageSequence is an interface that provides an easy to use and common interface for different image streams. Make sure you add the BoofCV module JavaCV to your project! JavaCV provides a wrapper around FFMPEG. If you don't include this package you will be stuck with just MJPEG files.

MediaManager media = DefaultMediaManager.INSTANCE;
SimpleImageSequence<Planar<GrayU8>> video = media.openVideo(fileName, ImageType.pl(3,GrayU8.class);

while( video.hasNext() ) {
  Planar<GrayU8> input = video.next();
  ... process ...
}

The following is a list of alternatives. Note that a built in MediaManager is not provided for these options.

Webcams

The recommended way to use webcams with BoofCV is with Webcam Capture. Webcam Capture is very easy to use and works out of the box on Linux, Windows, and MacOS. The BoofCV integration package only provides a single function for opening up a camera at a resolution as close to the one you requested as possible. You need to be sure that the BoofCV WebcamCapture module is included with your project.

Webcam Capture Example

Other Options:

BufferedImage

As previously mentioned BufferedImages can be converted into a BoofCV image using ConvertBufferedImage. Thus any library which can read a file into BufferedImages can be used with BoofCV.

Android

See Android Support for how to convert Android images into a BoofCV image.