A Tinder photo to remember.
Image: Flickr/Jake Stimpson
"Hell yeah," you think, as you upload a new photo to Tinder. "Let the matches roll in."
It's a snap of you in a picturesque farmer's field, arms raised above your head, which is thrown back as if to say, triumphantly, "I love the smell of cow turds!" You look cute, and you're in a nice location—what's not to like?
You get a few matches, but, alas, not many of them message you. Sorry, pal—you lost at Tinder roulette. But, hey, maybe you're not to blame. In fact, it could actually be that nice photo's fault. Maybe it's just not memorable enough.
A new deep learning system could help you out by predicting which of your photos people are most likely to remember. The system, a neural network called MemNet developed by researchers at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL), analyzes your photos and churns out a memorability ranking based on which photos humans thought were more memorable in tests. The system also overlays a "heat map" on your photos that highlights which parts of your photo people are likely to remember most.
"If we look at an image, just because we remember it, that doesn't remember that we remember all the little details of everything around it," Aditya Khlosa, the computer science PhD student who led the work, told me. "For example, if you saw an image of me, and then at a later time I changed the lighting slightly, you'd probably say that you've seen the image before. Because you're focusing on me, and that's what you remember. That's what we're trying to get out of our network: which parts of the image are important?"
If you'd like to give a few of your own dating photos a spin, you can do so with a demo site set up by the researchers.
According to a paper describing the system, which the researchers will present in Chile this week at the International Conference on Computer Vision, the system performed just as well as humans in tests. Moreover, as it turns out, photos in which a person's face is the focus tend to be more memorable than ones that feature a wider scene. "I was pleasantly surprised," Khlosa said of the team's results.
Khlosa and his colleagues' approach involves a clever trick called "fine tuning" that takes advantage of the power of neural networks—layers of digital "neurons" that run complex calculations on input data, like photos, and come closer and closer to a desired output as they "learn" when more data is introduced. It's a promising new technology that Elon Musk and his rich buddies recently backed with a $1 billion investment into a new non-profit research company.
Fine tuning involves mashing up parts of two different neural networks. The bottom layers of MemNet were taken from a system that learns to recognize objects and settings, and were trained on a huge number of photos. The top two layers, on the other hand, were trained on the memorability rankings that humans gave to photos in tests. When you give a photo to MemNet, the bottom layers do all the work determining whether the subject of a photo is a person or a dog, for example, or if they're sitting on a bicycle or in a chair. The top two layers then give the photo—once MemNet knows what it's "looking" at—an overall memorability ranking. To generate the heat map, the network analyzes the constituent parts of the image, instead of the whole thing together.
Watch more from Motherboard: Inhuman Kind
Using algorithms to determine how memorable objects, such as faces, are likely to be has been tried before, but never so successfully. Khlosa attributes this success to the CSAIL team's use of deep learning over more limited algorithms. MemNet isn't perfect, however.
When I fed a blank gray box to MemNet, the network gave it a "medium" score for memorability—the same ranking that it gave one of my Facebook profile photos. It gave the same ranking to a blank white box. This is because the network wasn't trained to recognize blank images, Khlosa said, and so it's a "limitation of the database," he explained.
But work on MemNet is hardly finished. Along with their paper, Khlosa and his colleagues are making their database of images, annotated with memorability scores, available for download. Other researchers can now use it to do their own work on AI and memorability, and perhaps even surpass MemNet's capabilities. The CSAIL team will also be working on making MemNet useful for things beyond just making your Tinder photo more memorable. For example, Khlosa said, he sees it as being useful in the classroom.
"When we were kids, we memorized a whole bunch of diagrams and the like, and it's a lot of work." Khlosa said. "If we could make scientific diagrams more memorable, visually, we could make the jobs of students and teachers much easier."
It could even be used to make text more interesting, Khlosa said. But how?
"I really don't know, as an honest answer," Khlosa told me. "But we got here, and we'll get further, hopefully in the next few years."