Difference between revisions of "Raspberry PI"

From BoofCV
Jump to navigationJump to search
m
m
 
(9 intermediate revisions by the same user not shown)
Line 1: Line 1:
[https://www.raspberrypi.org/ Raspberry PI's] (RPI) are low cost and very popular ARM devices hobby projects. BoofCV is very easy to set up and run on RPI's. Just compare my experience getting OpenCV and BoofCV up and running ([http://peterabeles.com/blog/?p=241 article]). Basically hours in a best case scenario vs minutes for BoofCV. Things change constantly so maybe it has gotten better since then. It also runs quite fast and well. Check out this these [[Performance:OpenCV:BoofCV|benchmark]] results.
[https://www.raspberrypi.org/ Raspberry PI's] (RPI) are low cost and very popular ARM devices hobby projects. BoofCV is very easy to set up and run on RPI's. Just compare my experience getting OpenCV and BoofCV up and running ([http://peterabeles.com/blog/?p=241 article]) in 2019. Basically hours in a best case scenario vs minutes for BoofCV. Things change constantly so maybe it has gotten better since then. It also runs quite fast and well. Check out this these [[Performance:OpenCV:BoofCV|benchmark]] results.


As is often the case with a RPI to get everything working well and working with your hardware (i.e. a camera) there will be some assembly required. Just as an example, let's say you want to use a RPI to scan QR codes using the RPI camera you need to do the following:
As is often the case with a RPI to get everything working well and working with your hardware (i.e. a camera) there will be some assembly required. Just as an example, let's say you want to use a RPI to scan QR codes using the RPI camera you need to do the following:


# [[#Replacing Default Java|Replace the JDK]]
# [[#How do I use the Raspberry PI Camera?|Setup the RPI Camera]]
# [[#How do I use the Raspberry PI Camera?|Setup the RPI Camera]]
# [[#Simple Example Application|Build and Run The Example Project]]
# [[#Simple Example Application|Build and Run The Example Project]]


Got an RPI running as a desktop machine? You should run the BoofCV demonstration applications! They mostly work. Cameras can be a bit of an issue though. RPI camera requires injecting special code that the demo application's don't have. USB cameras also work but USB is slow on a RPI. Here are the steps you need to follow
Got an RPI running as a desktop machine? You should run the BoofCV demonstration applications! They mostly work. Videos and cameras can be a bit of an issue though. Old mjpeg videos will run fast and mp4 video might work or might crash depending on how its encoded. RPI camera requires injecting special code that the demo application's don't have. USB cameras also work but USB is slow on a RPI. Here are the steps you need to follow


# [[#Replacing Default Java|Replace the JDK]]
# [[#Replacing Default Java|Replace the JDK]]
# Download and Run Demonstrations
# [[Applications|Download and Run Demonstrations]]
 
{{#ev:youtube|https://youtu.be/UdrCMB3josw|400|center}}




__FORCETOC__
__FORCETOC__


= Replacing Default Java =
= Simple Programming Example =
 
The default Java installation on Raspberry PI is not optimized for running on ARM and runs much, much slower than it should. You will need to download another version and use that.
 
# [https://www.bell-sw.com/pages/java-11.0.7/ Download JDK built for Raspberry PI]
# Decompress `tar -xzf jdk.tar.gz`
# Move to a location you like `sudo mv jdk-11 /opt`
# Add it to your path
 
= Simple Example Application =


Want a minimalist example? Want to scan for QR codes? Got you covered!
Want a minimalist example? Want to scan for QR codes? Got you covered!
Line 31: Line 23:


Instructions are in the README.MD and it starts by saying to follow the instructions here...  You do NOT need to build BoofCV to build that application. It will download all dependencies for you automatically.
Instructions are in the README.MD and it starts by saying to follow the instructions here...  You do NOT need to build BoofCV to build that application. It will download all dependencies for you automatically.
Here's the output that I see when I scan for QR Codes. Everything all together runs at 10 FPS and BoofCV takes about 20ms to process the image if it sees a QR and 10ms if it doesn't. Interesting to note that in this case frame capture is the performance bottle neck. This is on a Raspberry PI 3B+ using the Raspberry PI camera.
<pre>
0205 processed 640 x 480 in 20.11 (ms) FPS 10.2
  Found 1
    QR "LANDMARK16 1"
0289 processed 640 x 480 in 14.91 (ms) FPS 10.3
  Nothing detected
</pre>


= Building BoofCV =
= Building BoofCV =


You can build BoofCV the usual way, just make sure you use Java 11 or newer to do so and install the ARM optimized JDK! On a regular desktop it takes about 1 minute to build, but on a Raspberry PI it can take 10 minutes or so.
To use BoofCV on a Raspberry PI you do NOT need to build it. As was done in the example above, if you use Maven, Gradle, or any other modern build system you can automatically download all dependencies.
 
You can build BoofCV the usual way, just make sure you use Java 11 or newer installed! This will take about 10 minutes, which is about 10x slower than it is on a regular Desktop computer.


To build and install into your local Maven repository:
To build and install into your local Maven repository:
Line 56: Line 60:


<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
/gradlew -Dorg.gradle.java.home=/path/to/jdk-11/ demonstrations
./gradlew -Dorg.gradle.java.home=/path/to/jdk-11/ demonstrations
</syntaxhighlight>
</syntaxhighlight>


= How do I get webcams to work? =
= How do I get webcams to work? =


BoofCV uses [http://webcam-capture.sarxos.pl/ Webcam Capture] and in it's default configuration it doesn't work on Raspberry PI. The good news is that there is a simple fix that needs to be added to your source code to get it to run on Raspberry PI! https://github.com/sarxos/webcam-capture/wiki/How-To-Configure-Raspberry-Pi
Let's start by saying that USB webcams on a Raspberry PI 3B+ and earlier will be sluggish due to the limitations of USB on Raspberry PI. I've not yet tested a 4B yet. The Rapsberry PI camera will be much faster but the image quality is not  the best...


It should also be pointed out that you don't have to use any of built in tools. You just need a way to convert the image into format that BoofCV understands. Once you got the bytes for an image that is often "simple" if you understand image formats.
Out of the box BoofCV uses [http://webcam-capture.sarxos.pl/ Webcam Capture] due to its easy of use and in Webcam Capture's default configuration it doesn't work on Raspberry PI. The good news is that there is a simple fix that needs to be added to your source code to get it to run on Raspberry PI!


The [[#Simple Example Application|Simple Example]] has working code that contains the modification you need to do.
* [https://github.com/sarxos/webcam-capture/wiki/How-To-Configure-Raspberry-Pi Configuring Webcam Capture]
* [https://github.com/sarxos/webcam-capture/tree/master/webcam-capture-drivers/driver-raspberrypi More WebcamCapture Raspberry PI Goodness]
* [[#Simple Example Application| Example BoofCV Project Using Webcam Capture]]
* [https://groups.google.com/forum/#!topic/boofcv/utxMYaia7Ow BoofCV Forum Discussion]


After experimentation and TRFM it's recommended that you use <code>RaspividDriver</code> as the driver with Webcam Capture. With a Rapsberry PI camera it goes from 3 FPS to 10 FPS if you do. It still seems a bit slow to me. Let us know if you find a faster way to capture images.
* [https://github.com/Hopding/JRPiCam JRPiCam] uses raspistill to capture images. Not tried it but probably slower.
* [https://github.com/lessthanoptimal/libwebcam My fork of libwebcam.] Added a Java wrapper and added more controls. Should be more memory efficient. Not supported.
If there's another library you find that can capture images and is fast you should be able to wrap it and get the images into BoofCV.


= How do I use the Raspberry PI Camera? =
= How do I use the Raspberry PI Camera? =


The easiest way to access the camera is to treat it as a UVC device. This way WebcamCapture will recognize it and all the example code will work. Unfortunately, you have to tell your Raspberry PI to load the UVC driver each time you boot by doing the following ([https://raspberrypi.stackexchange.com/questions/10480/raspi-camera-board-and-motion Stack Exchange]):
The easiest way to access the camera is to treat it as a UVC device and use a library based on [https://en.wikipedia.org/wiki/Video4Linux V4L]. Unfortunately, you have to tell your Raspberry PI to load the UVC driver each time you boot by doing the following ([https://raspberrypi.stackexchange.com/questions/10480/raspi-camera-board-and-motion Stack Exchange]):


<syntaxhighlight lang="bash">
<syntaxhighlight lang="bash">
Line 86: Line 99:
</pre>
</pre>


/dev/video1 is the device associated with the Raspberry PI camera.
/dev/video1 is the device associated with the Raspberry PI camera. You can now access it using Webcam Capture if you follow the instructions in the section [[#How do I get webcams to work?| above]]
 
= Replacing Default Java =
 
In Raspbian prior to 2020 the default Java installed was not optimized for ARM processors and ran very slow. To get decent performance you needed to install a third-party build of Java which was done correctly. More recent builds of Raspbian seem to have fixed the issue, but you can follow the instructions below if you want to be safe.
 
# [https://www.bell-sw.com/pages/java-11.0.7/ Download JDK built for Raspberry PI]
# Decompress <code>tar -xzf jdk.tar.gz</code>
# Move to a location you like, e.g. <code>sudo mv jdk-11 /opt</code>
# Add it to your path, e.g. <code>PATH=/opt/jdk-11:$PATH</code>
 
This issue is also discussion in a comparison of [[Performance:OpenCV:BoofCV| BoofCV and OpenCV speed]].

Latest revision as of 07:01, 26 May 2020

Raspberry PI's (RPI) are low cost and very popular ARM devices hobby projects. BoofCV is very easy to set up and run on RPI's. Just compare my experience getting OpenCV and BoofCV up and running (article) in 2019. Basically hours in a best case scenario vs minutes for BoofCV. Things change constantly so maybe it has gotten better since then. It also runs quite fast and well. Check out this these benchmark results.

As is often the case with a RPI to get everything working well and working with your hardware (i.e. a camera) there will be some assembly required. Just as an example, let's say you want to use a RPI to scan QR codes using the RPI camera you need to do the following:

  1. Setup the RPI Camera
  2. Build and Run The Example Project

Got an RPI running as a desktop machine? You should run the BoofCV demonstration applications! They mostly work. Videos and cameras can be a bit of an issue though. Old mjpeg videos will run fast and mp4 video might work or might crash depending on how its encoded. RPI camera requires injecting special code that the demo application's don't have. USB cameras also work but USB is slow on a RPI. Here are the steps you need to follow

  1. Replace the JDK
  2. Download and Run Demonstrations



Simple Programming Example

Want a minimalist example? Want to scan for QR codes? Got you covered!

https://github.com/lessthanoptimal/ExampleWebcamRPI

Instructions are in the README.MD and it starts by saying to follow the instructions here... You do NOT need to build BoofCV to build that application. It will download all dependencies for you automatically.

Here's the output that I see when I scan for QR Codes. Everything all together runs at 10 FPS and BoofCV takes about 20ms to process the image if it sees a QR and 10ms if it doesn't. Interesting to note that in this case frame capture is the performance bottle neck. This is on a Raspberry PI 3B+ using the Raspberry PI camera.

0205 processed 640 x 480 in 20.11 (ms) FPS 10.2
  Found 1
     QR "LANDMARK16 1"
0289 processed 640 x 480 in 14.91 (ms) FPS 10.3
  Nothing detected

Building BoofCV

To use BoofCV on a Raspberry PI you do NOT need to build it. As was done in the example above, if you use Maven, Gradle, or any other modern build system you can automatically download all dependencies.

You can build BoofCV the usual way, just make sure you use Java 11 or newer installed! This will take about 10 minutes, which is about 10x slower than it is on a regular Desktop computer.

To build and install into your local Maven repository:

cd boofcv
./gradlew install

If you are running your Raspberry PI as a Desktop you can run the demonstration apps. Might also want to plug a camera in or setup your Raspberry PI camera as discussed below.

cd boofcv
./gradlew demonstrations
java -jar demonstrations/demonstrations.jar

Trouble Shooting:

If the `./gradlew` command fails and complains about Java 11 that means you didn't download Java 11 yet or your path is incorrect. One way to get around that is to force Gradle to use a specific JDK.

./gradlew -Dorg.gradle.java.home=/path/to/jdk-11/ demonstrations

How do I get webcams to work?

Let's start by saying that USB webcams on a Raspberry PI 3B+ and earlier will be sluggish due to the limitations of USB on Raspberry PI. I've not yet tested a 4B yet. The Rapsberry PI camera will be much faster but the image quality is not the best...

Out of the box BoofCV uses Webcam Capture due to its easy of use and in Webcam Capture's default configuration it doesn't work on Raspberry PI. The good news is that there is a simple fix that needs to be added to your source code to get it to run on Raspberry PI!

After experimentation and TRFM it's recommended that you use RaspividDriver as the driver with Webcam Capture. With a Rapsberry PI camera it goes from 3 FPS to 10 FPS if you do. It still seems a bit slow to me. Let us know if you find a faster way to capture images.

  • JRPiCam uses raspistill to capture images. Not tried it but probably slower.
  • My fork of libwebcam. Added a Java wrapper and added more controls. Should be more memory efficient. Not supported.

If there's another library you find that can capture images and is fast you should be able to wrap it and get the images into BoofCV.

How do I use the Raspberry PI Camera?

The easiest way to access the camera is to treat it as a UVC device and use a library based on V4L. Unfortunately, you have to tell your Raspberry PI to load the UVC driver each time you boot by doing the following (Stack Exchange):

sudo modprobe bcm2835-v4l2

It should now show up in the list of cameras if you type v4l2-ctl --list-devices. This is what I see on my system. I also have a webcam plugged in to USB that's why there are two devices:

mmal service 16.1 (platform:bcm2835-v4l2):
	/dev/video1

Logitech Webcam C930e (usb-3f980000.usb-1.1.3.1):
	/dev/video0

/dev/video1 is the device associated with the Raspberry PI camera. You can now access it using Webcam Capture if you follow the instructions in the section above

Replacing Default Java

In Raspbian prior to 2020 the default Java installed was not optimized for ARM processors and ran very slow. To get decent performance you needed to install a third-party build of Java which was done correctly. More recent builds of Raspbian seem to have fixed the issue, but you can follow the instructions below if you want to be safe.

  1. Download JDK built for Raspberry PI
  2. Decompress tar -xzf jdk.tar.gz
  3. Move to a location you like, e.g. sudo mv jdk-11 /opt
  4. Add it to your path, e.g. PATH=/opt/jdk-11:$PATH

This issue is also discussion in a comparison of BoofCV and OpenCV speed.