"What the Robots See", Input/Output, ME Departments, April 2006

input/output

by Jean Thilmany, Associate Editor

What the Robots See

Call them klutzes. Most robots are far too clumsy to navi-
gate around obstacles at high speeds.

That's because robots can't judge how far away objects are. They lack the depth perception that most of us take for granted when we maneuver through everyday life. But computer scientists at Stanford University recently came up with a way to help out the robots.

Expensive, large robots, it's true, can be equipped with a myriad of sensors that help them move through space and around objects. But now a distance-vision algorithm can give navigation capability to robots that are too small or too inexpensively built to carry all those sensors. All it takes is just one video camera, said Andrew Ng, an assistant professor of computer science at Stanford and one of the project's researchers.

To give robots depth perception, Ng and graduate students Ashutosh Saxena and Sung Chung designed software that can learn to spot certain depth cues in the still images that robots receive from the camera. The cues include variations in texture, edges, and haze. Lines that appear to be converging at their edges—like the sides of a path—indicate increasing distance; objects that appear hazy are likely farther away than clear objects.

Scientists at Stanford University are developing a way to give robots depth perception to maneuver around objects, like trees. Software takes distance cues from images like the one below.

To analyze such cues as thoroughly as possible, the software breaks images into sections and analyzes them both individually and in relationship to neighboring sections. This allows the software to infer how objects in the image stand relative to each other. The software also looks for cues in the image at varying levels of magnification to ensure that it doesn't miss details or prevailing trends—literally missing the forest for the trees.

With depth perception, robots judged distances with an average error of about 35 percent. For example, a robot perceived a tree 30 feet away as between 20 and 40 feet away, on average. A robot moving at 20 miles per hour and judging distances from video frames 10 times a second has ample time to adjust its path, even with this uncertainty, Ng said.

"The difficulty of getting visual depth perception to work at large distances has been a major barrier to getting robots to move and to navigate at high speeds," he said.

A radio-controlled car equipped with the algorithm recently drove autonomously for several minutes through a cluttered, wooded area before crashing, Ng said.

But he doesn't plan to stop his experiments there.

"I'd like to build an aircraft that can fly through a forest, flying under the tree canopy and dodging around trees," he said.