Building a Telepresence Robot
When building a robot you quickly work out that you have two choices with regards to controlling it: autonomous or some sort of remote control. We will develop both for Alexa M. We are going with remote control first because we are waiting for our ultrasonic mounting bracket to arrive from China.
As Alexa M has the Raspberry Pi camera fitted it makes sense to stream the video so we can have a view of what the robot is seeing. In effect a simple telepresence rover.
There are many different approaches for providing remote control to a robot (including wired, WiFi, Bluetooth, or RF). We wanted something wireless, with a Python API which could incorporate the video stream with minimal lag. That quickly narrowed things down and we chose control via WiFi.
Robot control via WiFi is pretty straight forward. You use a micro-framework like Bottle or Flask to set up the Pi as a web-server and then you can use your browser to access the associated web page. Well maybe it isn't that straight forward, but at least it is well documented. Streaming video to the same web page turned out to be a bit of a challenge - but not impossible. we were surprised that this wasn't a problem with an obvious solution given the numerous requests on the web for this functionality. The underlying issue seems to be that the Pi's camera outputs raw H.264, and what most browsers want is an MPEG transport stream. Given video was the tricky bit, we used this to decide which framework to use.
Video Streaming - The Options
The following is a list of the options that we came across when searching for a solution. No doubt there are many more, and if there are any we missed then let us know in the comments.
- picamera - was our first stop. It is s a pure Python interface to the Raspberry Pi camera module. Perfect! Except it doesn't do streaming. For anything else it is very good.
- bottle - is a fast, simple and lightweight WSGI micro web-framework for Python. It is distributed as a single file module and has no dependencies other than the Python Standard Library. The Raspberry Pi forums includes an example of how to stream video using bottle so this was definitely a contender. Electronut Labs provide a simple turn a LED on/off using bottle tutorial as well.
- flask - is another lightweight WSGI micro web-framework for Python. It is similar to bottle and you would probably choose flask over bottle if you had a more complicated application (over 1000 lines appears to be the consensus). Miguel has a tutorial on streaming video with flask and there is another guide provided by CCTV camera pros for the Raspberry Pi. Either flask or bottle would get the job done.
- Cayenne - helps you build a drag and drop web based dashboard for your IoT applications (i.e. Arduino and Raspberry Pi). It is pretty fancy but it cant do video streaming (yet).
- UV4L - was originally conceived as a modular collection of Video4Linux2-compliant, cross-platform drivers. It has evolved over the years and now includes a full-featured Streaming Server component. There is a module for single or dual Raspberry Pi CSI Camera boards but it is command line based and we would prefer a python API. At this stage there are easier options.
- pistreaming - provides low latency streaming of the Pi's camera module to any reasonably modern web browser. This is written by the same guy that did the picamera module, all the source code is provided and most importantly it is documented well enough for us to be able to modify the served page to do what we require. The video isn't as good as RPi-Cam-Web-Interface but there is no lag on our LAN. This is the option we ended up using.
To get the pistreaming solution to work you will need 3 files:
- index.html - the html code for the page that you are serving;
- server.py - the python code which serves up the video stream; and
These can all be cloned from the pistreaming repository. As a first step install the code by following the instructions at pistreaming. Once you have that up and working you can tweak it for your purposes.
RS_Server - a Video Streaming Python Class
To make streaming compatible with our robot class we have turned server.py into a server class. We have made a few other tweaks like inverting the camera since ours is mounted upside down. The print(server) command will display the URL where you can view the stream. The Server class is designed to be imported into another class and usage should be obvious from the class documentation and instructions at pistreaming.
We have also changed the index.html file in preparation for controlling the robot via the website, but we will cover this in a subsequent post.