Difference between revisions of "Android support"

From BoofCV
Jump to navigationJump to search
m
Line 1: Line 1:
BoofCV works without modification on Android systems, however you will need to convert Android specific image types into BoofCV image types. Support is provided for converting Bitmap, NV21, and YUV420 images.  Camera 2 API is actively supported while camera 1 API is unsupported but can still be used.
BoofCV's Android library provides support for converting between different image types and managing your android activity. If you use BoofCV to manager your activity it will select the camera, open the camera, create a thread pool, synchronizes data structures, handle the android life cycle (open/close the camera properly, stop threads), convert the YUV420 image, correctly align input pixels to screen pixels, and change camera settings upon request.


* [http://boofcv.org/javadoc/integration/boofcv/android/ConvertBitmap.html Convert Bitmap]
= Dependencies =
* [http://boofcv.org/javadoc/integration/boofcv/android/ConvertCameraImage.html Convert YUV_420_888]
* [http://boofcv.org/javadoc/integration/boofcv/android/ConvertNV21.html Convert NV21]


To use BoofCV in your project all you need to do is add the following line to dependencies in app/build.gradle
To use BoofCV in your project all you need to do is add the following line to dependencies in app/build.gradle
Line 10: Line 8:
api group: 'org.boofcv', name: 'boofcv-android', version: '0.31'
api group: 'org.boofcv', name: 'boofcv-android', version: '0.31'
</syntaxhighlight>
</syntaxhighlight>
No need to add jars manually, cut-and-paste BoofCV's source, sacrifice a chicken, or whatever weird stuff you guys keep on wanting to do. It's very simple. Well that's a little bit of a lie. You might also want to add the following to your app/build.gradle file since in some versions there's a conflict:


<syntaxhighlight lang="java">
<syntaxhighlight lang="java">
Line 22: Line 18:
</syntaxhighlight>
</syntaxhighlight>


= Trouble Shooting =


Does your code run much slower than the demo? Try switching the build to release. Click on "Build Variants" on the bottom left, then in the panel that pops up select "release" from the build variant.
= API Summary =
 
Support for Camera2 API was added May of 2018. The Camera 1 API is still available but not currently being updated. Static functions are provided for converting different image formats. If you wish to have BoofCV manage the Android activity you need to extend SimpleCamera2Activity or VisualizeCamera2Activity. The former will
only provide you access to control of the camera and access to its images. The latter handles converting
image data and assists in visualization. To use these activities you need to grant permission to the camera in your AndroidManifest.xml:
 
<syntaxhighlight lang="xml">
<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera2.full" />
</syntaxhighlight>
 
== SimpleCamera2Activity ==
 
To start the camera invoke ''startCamera} inside your Activity's onCreate() function.
<ul>
    <li>''selectResolution''
    <li>''onCameraResolutionChange''
    <li>''configureCamera''
    <li>''selectCamera''
    <li>''processFrame''
    <li>''onCameraOpened''
    <li>''onCameraDisconnected''
    <li>''changeCameraConfiguratio''
</ul>
To turn on and off verbose printing to Log you can toggle the SimpleCamera2Activity.verbose variable.
 
=== Selecting Camera / Resolution ===
 
By default BoofCV will select the back facing camera and the resolution which post closely matches the texture's width. NOTE: This behavior is changed if you extend VisualizeCamera2Activity. You can modify
this behavior by overriding one of the functions list above. For example, to change the resolution selection
behavior you will override ''selectResolution''.
 
=== Camera Settings ===
To customize the camera settings you need to override ''configureCamera''. If after opening the camera
you want to change the camera settings you need to then first call ''isCameraReadyReconfiguration''
and make sure the camera is in a state that it can be reconfigured. If that returns truen then you're
free to call ''changeCameraConfiguration'' which will eventually result in ''configureCamera''
being called again.
 
== VisualizeCamera2Activity ==
 
Extension of ''SimpleCamera2Activity'' which adds visualization and hooks for image
processing. Video frames are automatically converted into a format which can be processed by
BoofCV routines Optionally multiple threads can be generated so that video frames are
processed concurrently. The input image is automatically converted into a Bitmap image if
requested. If multiple threads are being used the user can toggle if they want visualization
to be shown if an old image finished being processed after a newer one.
 
Inside of onCreate() you need to call ''setImageType'' so that it knows what type of image to convert camera's YUV420_888 into. ImageType.single(GrayU8.class) for gray scale and ImageType.il(Gray8.class,3) for interleaved RGB. After that is done you will call ''startCamera(ViewGroup, TextureView)'' The textureview is optional and only needed if you want to display the camera preview directly.
 
<ul>
    <li><b>imageToView</b>: Matrix that converts a pixel in video frame to view frame</li>
    <li><b>displayView</b>: Where visualizations are rendered in.</li>
    <li><b>targetResolution</b>: Specifies how many pixels you want in the video frame</li>
    <li><b>stretchToFill</b>: true to stretch the video frame to fill the entire view</li>
    <li><b>bitmapMode</b>: Species how and if the bitmap should be drawn to the screen</li>
    <li><b>visualizeOnlyMostRecent</b>: If more than one thread is enabled should it show old results</li>
</ul>
 
== Converting Image Types ==
 
* [http://boofcv.org/javadoc/integration/boofcv/android/ConvertBitmap.html Convert Bitmap]
* [http://boofcv.org/javadoc/integration/boofcv/android/ConvertCameraImage.html Convert YUV_420_888]
* [http://boofcv.org/javadoc/integration/boofcv/android/ConvertNV21.html Convert NV21]


= Usage Examples =
= Usage Examples =
Line 30: Line 88:
A more detailed and working example for processing video images in Android can be found at the link below.  Short code snippets are included in this section.
A more detailed and working example for processing video images in Android can be found at the link below.  Short code snippets are included in this section.


Full Examples:
Full Working Examples:
* [[Example_Android_Gradient|Visualize Gradient]]
* [[Example_Android_Gradient|Visualize Gradient]]
* [https://github.com/lessthanoptimal/BoofCV/blob/SNAPSHOT/integration/boofcv-android/examples/video/app/src/main/java/org/boofcv/video/ExposureActivity.java Exposure Control]
* [https://github.com/lessthanoptimal/BoofCV/blob/SNAPSHOT/integration/boofcv-android/examples/video/app/src/main/java/org/boofcv/video/ExposureActivity.java Exposure Control]
Line 84: Line 142:


* [http://peterabeles.com/blog/?p=204 Demonstration Instructions]
* [http://peterabeles.com/blog/?p=204 Demonstration Instructions]
* [[Android_benchmark_app|Benchmark Wiki]]
 
= Trouble Shooting =
 
Does your code run much slower than the demo? Try switching the build to release. Click on "Build Variants" on the bottom left, then in the panel that pops up select "release" from the build variant.

Revision as of 07:31, 14 November 2018

BoofCV's Android library provides support for converting between different image types and managing your android activity. If you use BoofCV to manager your activity it will select the camera, open the camera, create a thread pool, synchronizes data structures, handle the android life cycle (open/close the camera properly, stop threads), convert the YUV420 image, correctly align input pixels to screen pixels, and change camera settings upon request.

Dependencies

To use BoofCV in your project all you need to do is add the following line to dependencies in app/build.gradle

api group: 'org.boofcv', name: 'boofcv-android', version: '0.31'
// conflicts with a Android stuff
configurations {
    all*.exclude group: "xmlpull", module: "xmlpull"
    all*.exclude group: "org.apache.commons", module: "commons-compress"
    all*.exclude group: "com.thoughtworks.xstream", module: "commons-compress"
}


API Summary

Support for Camera2 API was added May of 2018. The Camera 1 API is still available but not currently being updated. Static functions are provided for converting different image formats. If you wish to have BoofCV manage the Android activity you need to extend SimpleCamera2Activity or VisualizeCamera2Activity. The former will only provide you access to control of the camera and access to its images. The latter handles converting image data and assists in visualization. To use these activities you need to grant permission to the camera in your AndroidManifest.xml:

 <uses-permission android:name="android.permission.CAMERA" />
 <uses-feature android:name="android.hardware.camera2.full" />

SimpleCamera2Activity

To start the camera invoke startCamera} inside your Activity's onCreate() function.

  • selectResolution
  • onCameraResolutionChange
  • configureCamera
  • selectCamera
  • processFrame
  • onCameraOpened
  • onCameraDisconnected
  • changeCameraConfiguratio

To turn on and off verbose printing to Log you can toggle the SimpleCamera2Activity.verbose variable.

Selecting Camera / Resolution

By default BoofCV will select the back facing camera and the resolution which post closely matches the texture's width. NOTE: This behavior is changed if you extend VisualizeCamera2Activity. You can modify this behavior by overriding one of the functions list above. For example, to change the resolution selection behavior you will override selectResolution.

Camera Settings

To customize the camera settings you need to override configureCamera. If after opening the camera you want to change the camera settings you need to then first call isCameraReadyReconfiguration and make sure the camera is in a state that it can be reconfigured. If that returns truen then you're free to call changeCameraConfiguration which will eventually result in configureCamera being called again.

VisualizeCamera2Activity

Extension of SimpleCamera2Activity which adds visualization and hooks for image processing. Video frames are automatically converted into a format which can be processed by BoofCV routines Optionally multiple threads can be generated so that video frames are processed concurrently. The input image is automatically converted into a Bitmap image if requested. If multiple threads are being used the user can toggle if they want visualization to be shown if an old image finished being processed after a newer one.

Inside of onCreate() you need to call setImageType so that it knows what type of image to convert camera's YUV420_888 into. ImageType.single(GrayU8.class) for gray scale and ImageType.il(Gray8.class,3) for interleaved RGB. After that is done you will call startCamera(ViewGroup, TextureView) The textureview is optional and only needed if you want to display the camera preview directly.

  • imageToView: Matrix that converts a pixel in video frame to view frame
  • displayView: Where visualizations are rendered in.
  • targetResolution: Specifies how many pixels you want in the video frame
  • stretchToFill: true to stretch the video frame to fill the entire view
  • bitmapMode: Species how and if the bitmap should be drawn to the screen
  • visualizeOnlyMostRecent: If more than one thread is enabled should it show old results

Converting Image Types

Usage Examples

A more detailed and working example for processing video images in Android can be found at the link below. Short code snippets are included in this section.

Full Working Examples:

Converting Bitmap images

// Easiest way to convert a Bitmap into a BoofCV type
GrayU8 image = ConvertBitmap.bitmapToGray(bitmap, (GrayU8)null, null);
	
// If you are converting a sequence of images it is faster reuse a
// previously declare image and buffer
byte[] workBuffer = ConvertBitmap.declareStorage(bitmap, null);
ConvertBitmap.bitmapToGray(bitmap, image, workBuffer);
    	
// Convert back into a Bitmap
ConvertBitmap.grayToBitmap(image, bitmap, workBuffer);

// another less efficient way
bitmap = ConvertBitmap.grayToBitmap(image, Bitmap.Config.ARGB_8888);
    	
// Functions are also provided for multi-spectral images
Planar<GrayF32> color = ConvertBitmap.bitmapToMS(bitmap, null, GrayF32.class, null);

Converting YUV420 Images

Image image; // From camera in YUV 420 format
byte work[] = ConvertCameraImage.declareWork(image,null);
ConvertCameraImage.imageToBoof(image, ColorFormat.RGB, grayU8, work);

Converting NV21 Images

Data from camera previews in Camera 1 API is made available in the NV21 image format.

// from NV21 to gray scale
ConvertNV21.nv21ToGray(bytes,width,height,gray)

// from NV21 to YUV Multi-Spectral
ConvertNV21.nv21ToPlanarYuv(bytes,width,height,colorMS,GrayU8.class);

Besides converting images, VisualizeImageData is provided for visualizing

Android Apps

Android demonstration and benchmark applications are available for download from the google play store.

The demonstration visualizes several computer vision algorithms and provides a good estimate on what sort of runtime performance can be expected when using BoofCV on Android.

Trouble Shooting

Does your code run much slower than the demo? Try switching the build to release. Click on "Build Variants" on the bottom left, then in the panel that pops up select "release" from the build variant.