Tutorial Kinect

From BoofCV
Jump to: navigation, search

The Kinect a popular RGB-D sensor which provides both video and depth information. BoofCV provides support for the [1] driver directly through several helper function. The project already provides a Java interface. The Kinect is much easier to work with than stereo cameras and provides similar information. The major downside to working with a Kinect sensor is that they can't be use outdoors and have a more limited range.

To access helper function for Kinect in BoofCV go to the boofcv/integration/openkinect directory. Several useful examples and utilities are provided in the openkinect/examples directory, while the main source code is in openkinect/src directory. To use these functions, be sure to include the BoofCV_OpenKinect.jar in your project. This can be downloaded precompile or you can compile it yourself using ant.

Two classes are provided in the Kinect jar, StreamOpenKinectRgbDepth and UtilOpenKinect. StreamOpenKinectRgbDepth is a high level interface for streaming data from a kinect using OpenKinect. UtilOpenKinect contains functions for manipulating, reading, and saving Kinect data. The depth image is stored as an unsigned 16-bit gray scale image (GrayU16) and the RGB in any standard image format.

Related Examples:

Usage Example

Many more examples are provided in the openkinect/examples directory. Here is one showing typical usage.

 * Example demonstrating how to process and display data from the Kinect.
 * @author Peter Abeles
public class OpenKinectStreamingTest {

		// be sure to set OpenKinectExampleParam.PATH_TO_SHARED_LIBRARY to the location of your shared library!
		NativeLibrary.addSearchPath("freenect", OpenKinectExampleParam.PATH_TO_SHARED_LIBRARY);

	Planar<GrayU8> rgb = new Planar<GrayU8>(GrayU8.class,1,1,3);
	GrayU16 depth = new GrayU16(1,1);

	BufferedImage outRgb;
	ImagePanel guiRgb;

	BufferedImage outDepth;
	ImagePanel guiDepth;

	public void process() {
		Context kinect = Freenect.createContext();

		if( kinect.numDevices() < 0 )
			throw new RuntimeException("No kinect found!");

		Device device = kinect.openDevice(0);


		device.startDepth(new DepthHandler() {
			public void onFrameReceived(FrameMode mode, ByteBuffer frame, int timestamp) {
		device.startVideo(new VideoHandler() {
			public void onFrameReceived(FrameMode mode, ByteBuffer frame, int timestamp) {

		long starTime = System.currentTimeMillis();
		while( starTime+100000 > System.currentTimeMillis() ) {}
		System.out.println("100 Seconds elapsed");



	protected void processDepth( FrameMode mode, ByteBuffer frame, int timestamp ) {
		System.out.println("Got depth! "+timestamp);

		if( outDepth == null ) {
			outDepth = new BufferedImage(depth.width,depth.height,BufferedImage.TYPE_INT_RGB);
			guiDepth = ShowImages.showWindow(outDepth,"Depth Image");

		UtilOpenKinect.bufferDepthToU16(frame, depth);

//		VisualizeImageData.grayUnsigned(depth,outDepth,UtilOpenKinect.FREENECT_DEPTH_MM_MAX_VALUE);
		VisualizeImageData.disparity(depth, outDepth, 0, UtilOpenKinect.FREENECT_DEPTH_MM_MAX_VALUE,0);

	protected void processRgb( FrameMode mode, ByteBuffer frame, int timestamp ) {
		if( mode.getVideoFormat() != VideoFormat.RGB ) {
			System.out.println("Bad rgb format!");

		System.out.println("Got rgb!   "+timestamp);

		if( outRgb == null ) {
			outRgb = new BufferedImage(rgb.width,rgb.height,BufferedImage.TYPE_INT_RGB);
			guiRgb = ShowImages.showWindow(outRgb,"RGB Image");

		UtilOpenKinect.bufferRgbToMsU8(frame, rgb);


	public static void main( String args[] ) {
		OpenKinectStreamingTest app = new OpenKinectStreamingTest();