The current plan is to attempt to construct a 2D map based upon the features from omnidirectional stereo vision. I can locate edge features close to the ground plane quite well, but the trouble with edges is that they're not very unique. I could use the edge data, projected into cartesian coordinates, to begin building a local map, but after a short time the map would begin to degenerate.

So what's needed are more unique features rather than edges. These could be tracked between frames (data association), and I could then use an off-the-shelf graph based SLAM algorithm, such as TORO to build a map. At first I thought of using SIFT, which would be the obvious choice if I were an academic researcher, but there are software patent issues associated with that method that I'd rather not have to deal with. FAST corners would be nice, but the relatively low resolution caused by the mirror distortion means that this algorithm doesn't work well. But I can use the Harris corner features from "good features to track" which is already built into OpenCV. Having been an OpenCV refusenick for quite a number of years I'm now slowly growing to like it. Harris corners seem to work quite reliably, despite the low resolution.

Detecting the ground using an omnidirectional stereo vision system.

http://www.youtube.com/watch?v=JhNotSaBmnA

The green features in the centre mirror have been identified as being close to the ground plane.

Edge features close to the ground plane are detected by projecting all features from the centre mirror to the ground plane (the height of the camera is known), then reprojecting the ground features back into the image plane of the four peripheral mirrors. The reverse operation is then applied, and features within the centre mirror are compared. Features with small reprojection error must belong somewhere close to the ground plane.

This provides a convenient and general way of locating the ground, which does not depend upon unreliable texture, colour histogram or image segmentation methods.

A brief explanation about the new combined stereo and omnidirectional vision system on the GROK2 robot.

http://streebgreebling.blogspot.com/2010/02/combined-stereo-and-omnidirectional.html

Having tried the classical space carving/voxel colouring techniques, and found them wanting, I'm going to try simpler methods such as edge, line and motion detection. If the robot is stationary an observed moving object, like a person, should show up well and be easy to triangulate.

Looks like it's been a while since I last blogged here, so here's an update. In the last six months I've mainly been writing more stereo vision code. This was primarily for use with the Surveyor SVS, but I also wrote a version which runs on a PC under Linux.

http://code.google.com/p/libv4l2cam/

v4l2stereo was initially written as an easy way in which to test the stereo algorithm before transferring it to the blackfin, but later developed into a piece of software in its own right. You can see an example of the stereo disparities obtained with the Minoru webcam here:

http://www.youtube.com/watch?v=EUcLAarcj7U

The Minoru only has a short 6cm baseline, so the effective stereo range is not very great (probably less than two metres), but it works well on Linux. As is always the case with webcams the image capture is not synchronized, so if the cameras are moving quickly the delay does become a problem - but for most slow moving robots would be ok.

A recent nice feature of v4l2stereo is that it can be run in "headless" mode with no GUI output to the screen and also can stream the image over a network using gstreamer.

I also replaced Rodney's head with a simplified version which has the Minoru webcam mounted on it.

http://www.youtube.com/watch?v=gNRdcwrTOM0

As of December 2009 I've also been experimenting with omnidirectional vision. I saw one of the videos from Pi Robot a while back, and had been meaning to try out something similar using a Christmas tree decoration as a spherical mirror. This actually works very well, and I've now created a new project called Omniclops for this code, since I didn't want to mix it up with anything else.

http://code.google.com/p/omniclops/

Fortunately the geometry for a spherical mirror is pretty simple to deal with, and the results look promising. So promising in fact that it's a cause for regret that I didn't try doing this many years ago. I've been aware of this type of vision for at least a decade, but the mirrors always looked too exotic or expensive to be worth bothering with, and the idea of making a parabolic mirror by hand without milling machinery seemed like probably something which wouldn't be very successful.

Whilst fooling around with omnidirectional vision using a Christmas decoration a thought occurred to me. Could I somehow combine stereo vision with omnidirectional vision, so that objects could be ranged without needing to do structure from motion? At first I just thought of using a couple of mirrors with one of the stereo cameras, but then I thought why not just use a single camera with a wide field of view looking at multiple mirrors spaced some distance apart. This seems like a good way to do things for the following reasons:

- You only need a single camera

- There are no camera synchronisation issues

- There are no illumination/colour correction issues

- Ultra wide field of view compared to conventional stereo vision

- Very cheap to build

On the down side the resolution of the image within each mirror is rather low, but this probably isn't a major handicap. Also the geometry is more complex than for ordinary stereo vision, but not prohibitively so. I lashed up a prototype from aluminium and cardboard, using five mirrors made from Christmas decorations (carefully) sawn in half to make hemispheres. You can see the resulting effect like so:

http://www.youtube.com/watch?v=QIuljh7Piso

This is effectively the same as having five cameras with overlapping fields of view and fisheye lenses. Currently I'm thinking that this approach may be well suited to voxel coloring/space carving volumetric techniques, since it complies with the simple plane ordering constraint and the positions of the mirrors are known.

Workwise, I'm pretty much unemployed now - like a lot of software engineers at present - so I can work on this full time and see if I can get any useful volumetric modeling.

Calibrating the pan and tilt mechanism (again)

http://streebgreebling.blogspot.com/2009/07/pan-and-tilt.html

I've adapted older code to simplify this procedure and make it more integrated with the rest of the system.