Brands Want To Predict Your Behavior By Mining Your Face From YouTube Videos

If you were wondering how advertisers might one day exploit the massive troves of video being uploaded every day to social media sites, wonder no further.

The latest in creepy corporate surveillance comes from Chicago-based Mattersight Corporation and its "Predictive Video" system, which claims to analyze speech and facial expressions to "predict behavior based on the emotional state and personality style of any person in a video."

"Brands can now mine the personality data of a single user by analyzing video data publicly available online via social media channels" like YouTube and Vine, the company brags in a press release announcing its recent patent filing. The goal of the platform is to let companies predict (and thereby, manipulate) people's behavior by algorithmically studying their speech and facial expressions and developing personality profiles of individual users.

"Those insights enable us to make better, more precise predictions about how individuals feel about products and services, and what decisions they're likely to make in the future," Mattersight Chief Technology Officer Chris Danson wrote in the press release. The company says its mission is to "help brands have better conversations with their customers," which in this case apparently means "creepily scanning people's faces without their consent."

Technologies like predictive video seem to have recently caused advertisers to flock to video platforms, enabling them to mine previously-unattainable (and increasingly intimate) forms of human behavioral data. IBM has pitched its facial recognition and personality insights technologies as a way for retailers and marketing types to get real-time feedback on peoples' reactions to products while shopping. New social video platforms like Facebook Live also seem positioned to turn into a goldmine for facial profiling.

"[It's] creepy because it tries to read you from the inside. It literally tries to read your mind from your face," Adam Harvey, an artist and privacy researcher who has developed defenses against facial recognition, told Motherboard in an encrypted chat.

Mattersight says its system is intended to "improve the customer experience," but it's easy to see how the technology could be repurposed for mass-surveillance by police or military. At the 2014 Winter Olympics in Sochi, a similar technology called VibraImage scanned the facial expressions of visitors in order to give the Russian FSB the ability to "detect someone who appears unremarkable but whose agitated mental state signals an imminent threat," as the New York Times reported at the time.

U.S. law enforcement agencies are also very interested in the data companies like Mattersight aim to collect and analyze. The FBI has a massive biometrics database that was projected to contain more than 52 million faces by the end of last year, millions of which belong to US citizens who have never been suspected of a crime. In a public filing earlier this month, the FBI argued that it should be exempt from revealing whether or not someone is in the database, and should be allowed to keep the data indefinitely to predict future crimes.

But even if it remains in the hands of marketers, many people may not be cool with unknown companies profiling them by scanning every video they've ever appeared in on the internet—even if they may have tacitly agreed under some byzantine Terms of Service agreement.