How Facebook Is Helping Visually Impaired Users ‘See’ News Feed Photos

With more than 1 billion daily active users, Facebook has clearly done a pretty good job fulfilling its mission statement of making the world "more open and connected," but its work isn't quite done yet.

Today, April 5, Facebook begins rolling out a new feature called Automatic Alternative Text that helps blind and visually impaired users better understand photos that are posted to their News Feed. With Automatic Alternative Text, Facebook will audibly describe the contents of these photos using the built-in speech synthesizer of the iPhone.

The overall goal of the project, Facebook told me in March during a preview session at its New York office, is to ensure that Facebook doesn't leave behind the planet's 39 million blind and 246 million visually impaired people as the service becomes more and more focused on photos and videos. Assuming they sign up for an account, that is.

"When Facebook says [it wants to connect] everyone, we really do mean everyone," said Matt King, a Facebook accessibility engineer who joined the company in 2015 after a career building accessibility enterprise software at IBM. King, who is blind, knows firsthand the challenge of making technology accessible to people with disabilities. "You can't draw people to your product if it's a lousy experience. If it's a lot harder for people with disabilities they're simply not going to use it."

According to King, "tens of thousands" blind and visually impaired users browse Facebook with software known as a screen reader, which reads aloud app text as users navigate with a touchscreen or keyboard. (If you have an iPhone, you can enable a screen reader right now by telling Siri to "turn on VoiceOver." Similar software is typically available for other platforms in the settings or ease of use menus.)

To demonstrate the feature, King turned to Safari on a nearby MacBook Air, loaded up Facebook, then proceeded to scroll through the News Feed, finally settling on a photo from a friend named Clara. At this point, the robotic voice of OS X's screen reader software kicked in and said. "Link: Image. Image may contain sky, tree, outdoors."

"You get a little bit of a picture, right?" King asked. "It's not perfect, but it's something. You know she's doing outdoor landscape-type photography."

The feature is built upon Facebook's artificial intelligence research, specifically a branch called neural networks, a topic that Motherboard has previously covered. Here, everyday objects like cars, dogs, and ice cream are repeatedly placed in front of cameras that are connected to computers. Do this enough times, and Facebook is able to effectively "teach" its computers to recognize these objects in photos posted to the News Feed.

Facebook is keen to stress that this technology is still in its infancy and does not quite replicate the ability to actually see the photos your friends have posted. To King however, the feature represents an important first step in making Facebook more accessible to more people. "For me, we're not all the way there yet, but it's significant. It's way better than nothing.

To be sure, Facebook is hardly the only tech giant looking to improve the core experience for people who may be blind or visually impaired. Twitter in late March rolled out a feature that lets users manually add text descriptions to photos posted to the social network, enabling screen readers to then read back these descriptions. And Microsoft just about brought its Build developer conference to a standing ovation on March 30 when it showed a pair of experimental smartglasses that can identify everyday objects in near real time for the wearer.

"There's quite a bit of focus by the large tech companies that they have as many people using their products as possible," Joseph Lorenzo Hall, chief technologist at the Center for Democracy & Technology, told Motherboard, noting in particular the increased availability of accessibility tools in popular platforms like Wordpress that make websites more readable for people with disabilities. "The accessibility community is extremely vocal, and you don't get a lot of good PR by making these people feel bad!"