FYI.

This story is over 5 years old.

Tech

How Driverless Cars Will Keep Their Eyes on the Road

"Vision is our most powerful sense and driverless cars will also need to see."
SegNet visualization. Image: Alex Kendall

"Eyes on the road" is such a core proverb of safe driving that even driverless cars will have to obey it. Indeed, according to information engineer Roberto Cipolla, one of the biggest hurdles to overcome in the pursuit of self-driving cars will be replicating the visual acuity of a human driver.

"Vision is our most powerful sense and driverless cars will also need to see," said Cipolla, a professor based out of the University of Cambridge's Machine Intelligence Lab (MIL), in a statement. "But teaching a machine to see is far more difficult than it sounds."

Advertisement

Cipolla would know, given that he is developing this kind of road-watching software with fellow MIL researchers Vijay Badrinarayanan and Alex Kendall. Together, the trio have pioneered a visual system called SegNet, which was recently presented at the International Conference on Computer Vision in Santiago, Chile.

According to its website, SegNet is a "Deep Convolutional Encoder-Decoder Architecture for Robust Semantic Pixel-Wise Labelling," which is one seriously heavyweight logline.

But the phrase essentially boils down to a new method used to help machines to distinguish between all the complex visual parts of a typical drive, and to situate themselves in space. You can test it out for yourself by uploading pictures for the system to process on the SegNet demo page.

"Some of the big car manufacturers are using technologies like laser, radar, and cameras as well, and combining all these to try to create an autonomous vehicle," Kendall explains in the below video.

"What we're trying to do here is achieve all this with just vision. SegNet is a system that can label each pixel of an image with what it sees."

For instance, the typical road scene is an dynamic interaction between moving parts such as cars, pedestrians, cycles, road signs, hazards—the list goes on and on. To build Segnet, the team meticulous labeled images of all these different elements, pixel-by-pixel, and fed it to the program. After digesting about 5,000 of these images, SegNet has learned how to separate these categories fairly well by itself.

Advertisement

"It's remarkably good at recognising things in an image, because it's had so much practice," Kendall said. "However, there are a million knobs that we can turn to fine-tune the system so that it keeps getting better."

The team has also developed a localization system that can estimate the whereabouts and spatial orientation of a user from a single color image, with accuracy and coverage that Kendall says exceeds that of GPS.

Essentially, the idea is that if you were lost in New York like some latter day Kevin McCallister, you could take a picture and immediately know your location to within a few square meters, as well as the direction you were facing. These kinds of systems would have obvious applications for the navigation systems of self-driving cars.

At the moment, however, SegNet is still in its infancy, and Cipolla, Kendall, and Badrinarayanan expect it to be test-driven on domestic appliances, like autonomous vacuum cleaners, long before it hits the streets.

"It will take time before drivers can fully trust an autonomous car," Kendall said, "but the more effective and accurate we can make these technologies, The closer we are to the widespread adoption of driverless cars and other types of autonomous robotics."