This Algorithm Automatically Detects Vines That Don't Suck

How can we separate the truly creative Vines from the legions of half-assed, stupid, and bromidic ones?

Nov 20 2014, 7:00pm

​Image: Shutterstock

​You know what they say about history: First as tragedy, then as farce. Or, in the case of Vine: First as meme, then as art.

When Vine first emerged in 2012, it was merely a curiosity. Just two years later, Vine stars are a thing— they even hav​e their own pseudo-celebrity controversies—and high-profile film festivals are running​ Vine categories. But with millions of users Vine-ing at an incredible rate—Statista no​tes that five Vines are tweeted every second—not every 6-second clip is going to be a work of weighty genius.

How can we separate the truly creative Vines from the legions of half-assed, stupid, and bromidic ones?

Researchers from Yahoo Labs in Spain, as well as the University of Turin and Pompeau Fabra University, tackles this problem by building a computational model to automatically detect Vine videos that transcend the genre.

But is it art?​

The first step for the researchers was defining creativity, which they described as involving novelty and value in  ​their paper available on the arXiv preprint server. "Novelty," according to the researchers, can be defined as innovative and well-executed technique. Drawing on philosopher Immanuel Kant's 1790 treatise on aesthetics, Critique of Judgement, they defined "value" as involving sensory, emotional, and intellectual components.

To build the dataset of creative Vine videos they eventually used to construct the computer model, the researchers got Crowdflower workers to tag thousands of Vine videos as being creative or not based on their criteria. They ended up with nearly a thousand videos that could be considered "creative," with a high degree of agreement amongst the workers who did the tagging.

With a whack of creative Vines to draw from, the researchers then built their computational model of what makes a Vine video good.

Jessica Harmon's depiction of drug addiction in a 6-second clip won the drama category in Tribeca's Vine-sponsored Six Second Films category.

The sensory component of creativity was broken down into several categories and algorithmically analyzed: scene content, compositional technique, and filmmaking technique, which was further broken down into camera shake, quality of looping, use of stop motion effects, and more. 

The emotional component of creative Vine videos was calculated by measuring its visual quality (colour hues, level of detail, and more), and audio quality (loudness, key, dissonance, and rhythm). The intellectual value of a Vine was left out due to the lack of readily-available algorithms capable of computing it.

Finally, the novelty of a video was assessed based on how distant its features—based on the above criteria—were from other videos.

By now you're probably thinking, "Holy shit, this is a ludicrous amount of work to judge the kinds of internet videos I watch at 3 am while ripping bowls like Carl Sagan gazing into the cosmos." Me, too. But, finally, here we are.

This, then, is what makes Vine videos creative, according to a thorough computational analysis: "Features measuring order and uniformity correlate with creative videos, and creative videos tend to have warmer, brighter colors, and less frenetic, low-volume sounds. Also, they tend to be associated with pleasant emotions, and dominant, non-overwhelming, controllable emotions. Loop and Camera Shake features [...] also show high correlation with creativity."

When the researchers put their database of Vines identified as "creative" by Crowdflower workers to the test based on this model, they were able to predict a video's likelihood of being considered creative with 80 percent accuracy.

So, there you have it: The Vine videos most likely to be deemed creative loop well, have decent sound quality, use a nice color palette that brings pleasant emotions to mind, and use filmmaking techniques appropriately. Now, go forth and meme with the knowledge that you can also make art according to a computer model, if you have the technical know-how.