Here is a followup video to my previous blog entry. In this video, a number of independent behavioral threads are running to enable the robot to track and grasp the green balloon. Whenever the balloon is grasped, the robot turns its attention to the red balloon. When the green balloon is released, tracking turns again to it and the red balloon is ignored. I use RoboRealm to do the green/red tracking. There is a sonar sensor on the inside of the left hand that tells the robot when something is ready to be grasped. It can also do this using vision alone along with some trigonometry, but the result is more reliable when using the sensor.

--patrick

http://www.pirobot.org