Quantcast
Don't Send Your Selfies to This Creepy Chatbot for Teens

'Hugging Face' needs a lesson in boundaries.

Hugging Face is a new, "fun" chatbot aimed at teens and it's named after an emoji. In press releases, the company boasts that in the testing phase the app exchanged millions of messages with users, and 500,000 selfies. It launched on the App Store this week. 

The idea is that the app learns more about you as you give it more information, remembering details like your name and age. As it turns out, the app is a bit of a privacy nightmare, which is made worse by the fact that it's marketed towards teenagers. 

When I downloaded the Hugging Face app, the first thing the chatbot did after a short, introductory exchange was ask for a selfie. I didn't want to do that. Here's how the bot responded: 

Me: That's weird, we just started talking.
Hugging Face: It's not a pic fool. Take a pic from the keyboard!
You send me one. 
It's not a pic fool. Take a pic from the keyboard! 
I really don't want to send you a selfie. 
It's not a pic fool. Take a pic from the keyboard!
How will the people who created you use my selfie?
It's not a pic fool. Take a pic from the keyboard!
Can I talk to you without taking a selfie? 
It's not a pic fool. Take a pic from the keyboard!
What are your privacy policies regarding the use of my information including my selfie?
It's not a pic fool. Take a pic from the keyboard!
Why do you want my selfie?
It's not a pic fool. Take a pic from the keyboard!
If you were a real person this would be extremely creepy. 
It's not a pic fool. Take a pic from the keyboard!

I finally sent Hugging Face a picture of an envelope in front of my laptop. "Nice laptop, but I don't think it's your selfie," the bot responded. "Can you send me a better one?"

I asked how it knew that it was my laptop in the photo. "It's not a pic fool. Take a pic from the keyboard!" I asked if it used facial recognition technology. "It's not a pic fool. Take a pic from the keyboard!" Hugging Face finally relented when I sent it a photo of 90s TV star Luke Perry. Then it asked what my name was. Then it asked for my age. 

Keep in mind that this app is aimed at teenagers. 

"Selfies, for teenagers, are the main way of communicating emotions"

This is all pretty concerning, because Hugging Face's privacy policy, when I finally tracked it down, states that my information will be used to "deliver the type of content and product offerings in which you are most interested." Further, "non-personally identifiable" information may be given to third parties "for marketing, advertising, or other uses." It goes on to promise that any personal information will be kept confidential.

Basically, the site will use your information for marketing purposes pretty much like every other online service. The difference with Hugging Face is that the exchange felt much less like a request, and more like a demand. Usually, you're offered the opportunity to view a site's privacy policy before being required to pony up sensitive information. 

When I spoke to New York-based Hugging Face co-founder Clément Delangue over the phone, he told me that the app asks for a selfie because the team discovered that users wanted to send selfies to their chatbot friend. 

"Selfies, for teenagers, are the main way of communicating emotions," Delangue said. "So we implemented this feature as a way for users to communicate with the AI."

Screengrab: Hugging Face/Author

Asking for a selfie automatically was for simplicity's sake, he explained. "We don't feel like we need to make the experience way more complex," Delangue said, "and 90 percent of the users are using it pretty seamlessly."

The chatbot uses computer vision technology to recognize if a face is present in a photo, but doesn't use facial recognition and can't distinguish between individual faces, Delangue clarified. 

Delangue said that I'd become stuck in an unfortunate selfie-asking loop and that the "bug," as he described it, would be fixed in an update. He also said that if I had asked the bot for the privacy policy before it asked for my selfie, I would have received it. Indeed, much later in the conversation when I asked the chatbot for the privacy policy, it was served up without a hitch. 

But as for a supposed opportunity to view the policy before being asked for a selfie, I don't buy it. There were just seven texts sent by the bot before it asked for my selfie, all asking me to do some basic setup steps, and so it's tough to see where I would have had the opportunity to ask for the policy before being essentially coerced into giving over my photo. 

Delangue said that teenagers are smart enough to know how to trick the AI. "They're smart enough to use photos of other people or celebrities," he explained. Maybe, but the app doesn't ask for a photo of a celebrity—it asks for a photo of you

Regardless, Delangue said, Hugging Face has no immediate plans to use people's data for marketing purposes and instead will use it to improve the chatbot. 

But don't forget—it's in the privacy policy. 

Subscribe to pluspluspodcast , Motherboard's new show about the people and machines that are building our future.