Facial Recognition Apps Are Leaving Blind People Behind

Biometric logins like facial recognition are often discussed as being the real alternative to traditional passwords. But as companies develop apps that verify identity with a snapshot of the user's face, few are considering whether that method is accessible to people with visual impairments.

According to the World Health Organization, there are 285 million people in the world with visual impairments of some sort, 39 million of which are blind. This is a large segment of the world that could potentially be left behind by facial recognition logins, especially if biometrics become standard.

User your computer or smartphone's camera, facial recognition software identifies a user's face when logging in by comparing your real time image with a headshot it has stored when you set it up. Facial biometrics for logging in or verifying identity are becoming more common with products from major companies like Windows Hello, Intel True Key, and Mastercard's pay-by-selfie verification. Startups like BioID and FacePhi that create face scanning apps for accessing things like banking apps. But do they work for people with sight issues?

In March, researchers from the University of Surrey, University Group for Identification Technologies (GUTI), and Carlos III University of Madrid published their exploratory research into how people with poor vision use facial biometrics for identity verification. The study found that people with visual impairments struggle with taking selfies, a key feature for many biometric solutions that use the face to verify the user. This is mainly because the poor positioning of the camera leads to the face being blurry or off-center.

The team is also building the first database of blind people's faces for further research. "The fact that there isn't a database for blind subjects I think is quite significant, because possibly people haven't looked at how the technology is used for visually impaired subjects," study author Dr. Norman Poh told Motherboard.

The researchers carried out a series of different experiments with the subjects, such as taking a selfie without help, taking one with audio cues from a computer that detected the location of the head, or taking one with help from a supervisor. In many cases, the audio feedback proved the most helpful.

Audio prompts used by the researchers would detect and guide the subject's head into an in-focus shot that captured a frontal view of the face. In this case, the researchers used a beeping sound that would increase in frequency and volume as the subject's head moved closer into the correct position.

In their conclusion, the researchers called for more "careful and considerate design" of facial recognition applications to insure they are accessible for people with vision disabilities.

For instance, Poh and his colleagues suggested using "alignment-free" algorithms in facial recognition. Unlike traditional alignment techniques that detect certain facial features together, the alignment-free approach "relies on key points like the eyes, nose, and mouth, because these are salient features," explained Poh. "They can localize the interesting feature points. The algorithm actually matches the corner of the eyes or nostril to nostril and the comparison can be made much more reliably. It's done without any alignment of the picture."

The researchers used the Scale-Invariant Feature Transform (SIFT) algorithm, which can detect key points in an image that has several different features, and is "resistant to occlusion, scale and orientation changes."

But despite the growing availability of biometric identification options, many of which use the face as well as fingerprint or voice recognition, the issue of accessibility for the visually impaired hasn't really come up for many companies.

Cognitec, which develops facial recognition software, told Motherboard that this was an unusual inquiry. "I have really not thought about that in all this time," said marketing manager Elke Oberg. "As far as I know in our products, there aren't really any particular features or capabilities that take that into account."

Oberg said that although companies like Cognitec are developing this technology, it's the clients that implement it in their apps that need to consider how it can be made more accessible for their user base. "We would give these people the actual algorithm for the face recognition, but it's not a product that we develop on our own," she said.

Tom Grissen, CEO of biometrics firm Daon, said the firm has developed technology to provide audio feedback for users on positioning their face, but this hasn't been adopted among its customer base. "I don't know if anybody has put it into a standard product feature that they have released to the market," said Grissen. "I know they're all looking at it and we're working with them to find the best techniques."

Spanish biometrics startup FacePhi, which targets its products at mobile banking, said they have nothing in the works yet. "Right now we don't have anything to help guide a blind person to register an authentication," said Tania Martínez, executive assistant at FacePhi. "We're studying the possibility and I'm sure in the near future we're going to start with the implementation of this."

Rather than using audio cues, the startup is considering using vibration to guide the user, she added. Vibration prompts could be used by a person in loud areas where they may not hear audio. Similar to the audio cue, the vibration may increase in intensity as the face is positioned closer to the center.

Another potential way to identify users, added Grissen, is with "liveness" techniques that would require the user to nod at the camera, or move in a way that proves they're real, rather than say holding up a photograph of someone's face. This combined with an audio prompt could help a visually impaired user through the process quicker. We've already seen Amazon patent a similar liveness technique recently by suggesting a wink.

"This is a typical user interface and design issue, not so much a technology issue," said Ho Chang, CEO of BioID, another biometric tech startup.

Poh, the study author, said that while his research focused on the visually impaired, blindness is just one of the many issues to consider when designing apps and software, and accessibility for people with disabilities needs to factor more into the decision-making behind technology's user experience. "If you want to roll out technology," he said, "you have to consider all population of users."