Humans Are Really Good at Facial Recognition

Over the past year or so we’ve seen machines get creepily good at picking out faces in digital images. A couple of months ago, for instance, we saw face-scanning screens at Tesco supermarkets that could determine a viewer’s age and gender so as to better target ads to them. And that had nothing on an existing technology in Japan, which also registers a subject’s facial bone structure and purchasing history to identify individual customers and track their shopping habits.

Then, of course, there’s Facebook’s photo tagging system, which seems to get more accurate each time you upload an image. When I posted a particularly dark and grainy photograph of myself showing off the Ryan Gosling colouring book I got for Christmas, it even prompted me to tag the sketchy outline of the film star’s face. I may be a really good colour-inner, but that’s still eerily good face detection work.

But while algorithms are good at picking out faces and in some cases even matching them up to individuals, they’ve got nothing on our own facial recognition abilities (and we're also not so easily confused by distracting make-up or funky face-scrambling glasses).

A study published this week in the journal PLOS ONE tested people’s ability to identify images of faces from a very low-quality source: the reflection in someone else’s eye. The study was conducted by psychologists Rob Jenkins from the University of York and Christie Kerr from the University of Glasgow, both in the UK.

The purpose of their research was to see how this sort of facial recognition could be used to identify perpetrators in criminal investigations where victims are photographed. For example, an image taken from a child porn case could inadvertently reveal a reflection of the person behind the camera or other "bystanders." As the researchers explained:

"Cameras are routinely seized as evidence during criminal investigations. Images of people retrieved from these cameras may be used to piece together networks of associates, or to link individuals to particular locations. In particular, it may be desirable to identify the photographer, or other individuals who were present at the scene but were not directly captured in the photograph. Bystander identification may be especially important when the images record criminal activity, as when hostage takers or child sex abusers photograph their victims."

They took high-res digital photos of eight volunteer subjects, and zoomed in to uncover the “hidden” images of bystanders captured in the reflection of their eyes. The effect is similar to that when you try to take a photo through a window and end up with a reflected picture of yourself and your surroundings, but on a much smaller scale.

The contrast of the bystander images was adjusted a little with PhotoShop (as would likely happen in an investigation), and participants were asked to say whether the person shown matched a standard photo of the same person or someone who looked similar to them. Participants correctly matched the images 71 percent of the time when they weren’t already familiar with the subjects, and 84 percent when they were (these participants shared university classes with the subjects). In a second test, they also found that people who were familiar with one of the individuals could successfully identify and name them among a line-up of other faces they didn’t know 90 percent of the time.

That’s quite impressive considering the very low quality of these kind of "bystander" images. The researchers set them up so that the photographic subject was looking at five other people, including the photographer. They explained that the subject’s face, excluding hair, accounted for an area of around 12 million pixels. Of that, just 0.5 percent was taken up by the iris of their eye. People captured in the iris' reflection took up just 322 pixels—0.003 percent of the area of the subject’s own face.

“A face image that is recovered from a reflection in the subject’s eye is about 30,000 times smaller than the subject’s face,” said Jenkins in a statement. “Our findings thus highlight the remarkable robustness of human face recognition, as well as the untapped potential of high-resolution photography.”

Of course, viable photographs won’t always be available in criminal investigations—the pictures need to show the subject looking more or less straight at the camera, must have their face in focus, and be reasonably well-lit in order to extract this kind of corneal reflection—but the researchers pointed out that cameras are increasing in spec all the time, which means the quality of is there.

“However, as the current study emphasizes, the extracted face images need not be of high quality in order to be identifiable,” they concluded. “For this reason, obtaining optimal viewers—those who are familiar with the faces concerned—may be more important than obtaining optimal images.”