All the Feels: A Scientific Ranking of Emoji Sentiment

Researchers have created the Emoji Sentiment Ranking, a qualitative list of 751 emojis ranking how positive, negative, or neutral each image is.

What's in an emoji? A

by any other pictogram would be as sweet. Yet we still struggle sometimes to suss out exactly what an emoji is meant to convey.

There's plenty of debate over the precise meaning of different emojis as they continue to infiltrate our daily communication, and we might not be able to write an emoji dictionary just yet. But what about the general sentiment of these images? Sure there's a way to qualify whether a particular emoji is generally positive or generally negative.

That's what researchers did in a paper published in PLOS One Monday, where they introduce the Emoji Sentiment Ranking, a qualitative list of 751 emojis indicating how positive, negative, or neutral each image is. It may seem superfluous to qualify tiny pictograms of see-no-evil monkeys in this way, but as emoji usage continues to increase, it only makes sense to start treating them as a paralanguage and figuring out just what it is we're trying to say.

So how exactly do you make a ranking of something that seems so personal and subjective? To start, the group of four researchers scraped 1.6 million tweets in 13 European languages from a two-year period. Of those tweets, 70,000 contained one or more of 969 different emojis. But to get a slightly better sample size, the researchers decided to ditch any emojis that were used in fewer than five tweets, leaving them with 751 emojis.

They then had all of the tweets parsed by native speakers and labelled as either negative, positive, or neutral. Of course, that's not a highly scientific way of determining the sentiment behind a smiley face or hand gesture, so they brought in some math. They plugged the data provided by the native speakers into a series of formulae to calculate how positive, negative, or neutral each emoji was (which they dubbed a "sentiment score"), as well as how variable these labels were (which they dubbed its "neutrality" level).

A plot showing all of the emojis based on their sentiment score and neutrality level. Bubbles to the left of zero are more negative, while those to the right are more positive. Image: PLOS One

That means that an emoji like the yin yang symbol, which is pretty much the definition of neutral, is right in the middle of the sentiment score and very high on the neutrality level, because everyone agrees on what it means. Meanwhile a crying face can sometimes be considered positive in the context of the right tweet, and other times very negative, leaving it with a fairly neutral sentiment score, but a low neutrality level:

A close up of some of the negative (A), neutral (B), and positive (C) emojis from the plot above.

A lot of the results are pretty intuitive: the wrapped present emoji, for example, has one of the highest sentiment scores of 0.76, which is almost entirely positive (who doesn't like getting a present?). The crying cat face has a sentiment score of -0.37, almost all negative (poor kitty).

Others are a bit surprising. The straight line mouth face, which is literally called "neutral face," actually has a pretty negative sentiment score, as does the police officer emoji, and the bento box emoji. The most positive emoji, with a sentiment score of 0.96 is weirdly a vertical dashed line. But some of the emojis were used as few as five times, making the sample size too small for a definitive analysis.

The researchers found other interesting results in their study by cross-examining their data. Tweets with emojis were more likely to be interpreted the same way (positive, negative, or neutral) by multiple people than ones with no emoji, for example.

"The presence of emojis has a positive impact on the emotional perception of the tweets by humans," the authors wrote in the paper. "After all, this is probably the main reason why they are used in the first place."

The lexicon is by no means the definitive ranking of emoji sentiment, but through their analysis, the researchers have created an open source library that other researchers can now build on. It's similar to what's been done with other language databases, like SentiWordNet, a lexicon of sentiment rankings for English words, which researchers use for opinion analysis. With 1,281 emoji characters in the current Unicode version, emojis are continuing to expand the ways we can communicate, so it's natural that researchers interested in language will want to start tracking how we're annotating and supplementing our messages using these little cartoon faces.