Android support

BoofCV's Android library provides support for converting between different image types and managing your android activity. If you use BoofCV to manager your activity it will select the camera, open the camera, create a thread pool, synchronizes data structures, handle the android life cycle (open/close the camera properly, stop threads), convert the YUV420 image, correctly align input pixels to screen pixels, and change camera settings upon request.

Dependencies

To use BoofCV in your project all you need to do is add the following line to dependencies in app/build.gradle

api group: 'org.boofcv', name: 'boofcv-android', version: '0.31'

// conflicts with a Android stuff
configurations {
    all*.exclude group: "xmlpull", module: "xmlpull"
    all*.exclude group: "org.apache.commons", module: "commons-compress"
    all*.exclude group: "com.thoughtworks.xstream", module: "commons-compress"
}

API Summary

Support for Camera2 API was added May of 2018. The Camera 1 API is still available but not currently being updated. Static functions are provided for converting different image formats. If you wish to have BoofCV manage the Android activity you need to extend SimpleCamera2Activity or VisualizeCamera2Activity. The former will only provide you access to control of the camera and access to its images. The latter handles converting image data and assists in visualization. To use these activities you need to grant permission to the camera in your AndroidManifest.xml:

 <uses-permission android:name="android.permission.CAMERA" />
 <uses-feature android:name="android.hardware.camera2.full" />

SimpleCamera2Activity

To start the camera invoke startCamera} inside your Activity's onCreate() function.

• selectResolution
• onCameraResolutionChange
• configureCamera
• selectCamera
• processFrame
• onCameraOpened
• onCameraDisconnected
• changeCameraConfiguratio

To turn on and off verbose printing to Log you can toggle the SimpleCamera2Activity.verbose variable.

Selecting Camera / Resolution

By default BoofCV will select the back facing camera and the resolution which post closely matches the texture's width. NOTE: This behavior is changed if you extend VisualizeCamera2Activity. You can modify this behavior by overriding one of the functions list above. For example, to change the resolution selection behavior you will override selectResolution.

Camera Settings

To customize the camera settings you need to override configureCamera. If after opening the camera you want to change the camera settings you need to then first call isCameraReadyReconfiguration and make sure the camera is in a state that it can be reconfigured. If that returns truen then you're free to call changeCameraConfiguration which will eventually result in configureCamera being called again.

VisualizeCamera2Activity

Extension of SimpleCamera2Activity which adds visualization and hooks for image processing. Video frames are automatically converted into a format which can be processed by BoofCV routines Optionally multiple threads can be generated so that video frames are processed concurrently. The input image is automatically converted into a Bitmap image if requested. If multiple threads are being used the user can toggle if they want visualization to be shown if an old image finished being processed after a newer one.

Inside of onCreate() you need to call setImageType so that it knows what type of image to convert camera's YUV420_888 into. ImageType.single(GrayU8.class) for gray scale and ImageType.il(Gray8.class,3) for interleaved RGB. After that is done you will call startCamera(ViewGroup, TextureView) The textureview is optional and only needed if you want to display the camera preview directly.

• imageToView: Matrix that converts a pixel in video frame to view frame
• displayView: Where visualizations are rendered in.
• targetResolution: Specifies how many pixels you want in the video frame
• stretchToFill: true to stretch the video frame to fill the entire view
• bitmapMode: Species how and if the bitmap should be drawn to the screen
• visualizeOnlyMostRecent: If more than one thread is enabled should it show old results

Processing Images

You must implement processImage() and typecast the image it provides into the format you specified earlier

Visualization

Override onDrawFrame() and invoke the super.

Converting Image Types

• Convert Bitmap
• Convert YUV_420_888
• Convert NV21

Usage Examples

A more detailed and working example for processing video images in Android can be found at the link below. Short code snippets are included in this section.

Full Working Examples:

• Visualize Gradient
• Exposure Control
• QR Code Detect and Visualize

Converting Bitmap images

// Easiest way to convert a Bitmap into a BoofCV type
GrayU8 image = ConvertBitmap.bitmapToGray(bitmap, (GrayU8)null, null);
	
// If you are converting a sequence of images it is faster reuse a
// previously declare image and buffer
byte[] workBuffer = ConvertBitmap.declareStorage(bitmap, null);
ConvertBitmap.bitmapToGray(bitmap, image, workBuffer);
    	
// Convert back into a Bitmap
ConvertBitmap.grayToBitmap(image, bitmap, workBuffer);

// another less efficient way
bitmap = ConvertBitmap.grayToBitmap(image, Bitmap.Config.ARGB_8888);
    	
// Functions are also provided for multi-spectral images
Planar<GrayF32> color = ConvertBitmap.bitmapToMS(bitmap, null, GrayF32.class, null);

Converting YUV420 Images

Image image; // From camera in YUV 420 format
byte work[] = ConvertCameraImage.declareWork(image,null);
ConvertCameraImage.imageToBoof(image, ColorFormat.RGB, grayU8, work);

Converting NV21 Images

Data from camera previews in Camera 1 API is made available in the NV21 image format.

// from NV21 to gray scale
ConvertNV21.nv21ToGray(bytes,width,height,gray)

// from NV21 to YUV Multi-Spectral
ConvertNV21.nv21ToPlanarYuv(bytes,width,height,colorMS,GrayU8.class);

Besides converting images, VisualizeImageData is provided for visualizing

Android Apps

Android demonstration and benchmark applications are available for download from the google play store.

• Demonstration

The demonstration visualizes several computer vision algorithms and provides a good estimate on what sort of runtime performance can be expected when using BoofCV on Android.

• Demonstration Instructions

Trouble Shooting

Does your code run much slower than the demo? Try switching the build to release. Click on "Build Variants" on the bottom left, then in the panel that pops up select "release" from the build variant.