Fifty Percent of Mechanical Turk Workers Have College Degrees, Study Finds

Since 2005, researchers and businesses have increasingly relied on Mechanical Turk (MTurk). Using the online marketplace owned by Amazon, companies can request for individuals to complete bite-sized pieces of work, called Human Intelligence Tasks (HITs), in exchange for a couple cents each.

The kinds of tasks workers are asked to complete can vary widely, from transcribing audio files and editing text, to identifying what a photo depicts, among many other things. Oftentimes, the tasks are taken out of context, and it's not clear what purpose they may serve.

Essentially, individuals do tasks that are still difficult for computers to complete but are easy for humans. A 2015 World Bank report estimated that there are about half a million workers on Mechanical Turk, although not all of them are active.

MTurk is particularly attractive for academics and nonprofits who often are too cash strapped to conduct a national survey. The problem is, as a new Pew Research Center report out today shows, the workers on Mechanical Turk aren't representative of the general American population. Even worse, they're often severely underpaid for the work that they do.

Pew's survey of 3,370 individuals found that Mechanical Turk workers are younger and more educated than the average American. 51 percent of respondents said that they had a college degree, compared to 36 percent of working adults over 18 in the general population. 88 percent are under 50 years old, compared with 66 percent of employed adults nationally.

In terms of race and income, MTurk workers also don't represent national averages. 77 percent of them reported identifying as white, and 74 percent said that they live in households making $75,000 dollars or less a year, compared to 47 percent of adult workers in the general population.

The most alarming results of the study showed that MTurk workers generally reported earning less than minimum wage. 52 percent of workers in Pew's sample said they earned less than $5 dollars an hour. Only 8 percent said they earned $8 dollars an hour or more.

If MTurk workers only completed tasks for extra cash, the fact that they were paid so little might matter less, but the Pew study showed that this isn't the case. Almost two-thirds of Pew's sample said that they perform tasks on Mechanical Turk everyday.

Only 25 percent of respondents said that they relied on Mechanical Turk for all or most of their income, but it's still problematic to discover that many of them are being paid below the hourly federal minimum wage of $7.25 for their work. Older workers seem to bear the brunt of the problem. Those 50 or older were more likely than younger workers to earn low wages.

"It feels unfair that requesters can get away with paying less than minimum wage, meanwhile, we get taxed on that income pretty harshly depending on how much you turk and other sources of income you earn," Reddit user and MTurk worker HydroponicFunBags said.

Similarly to other gig economy workers, Mechanical Turk workers are considered to be independent contractors and not employees of Amazon.

Another concern for researchers is that MTurk workers may simply be overexposed to academic surveys, and thus conditioned to answer questions in certain ways. A 2014 study that polled 291 MTurk workers found that the median worker reported participating in 300 academic studies on MTurk, and 20 in the previous week alone.

Despite unfair pay and potentially skewed results, if academics were to rely less on MTurk, they might be taking some of the most fairly paid gigs away.

"I think that around 50 percent of academic requesters try to pay fairly, which is more than the requester pool in general; meaning a higher percentage of academic requesters care about paying fairly compared to requesters who aren't posting surveys," Reddit user auralgasm, who said he works on MTurk full time, told me for a previous article.

Many requesters on the platform, academic or otherwise, are impossible to identify, making it hard to regulate issues like fair pay. Pew examined the requesters on Mechanical Turk for a week during December 2015, and it was unable to identify what category 32 perecent of them even fell into.

Reddit user and Mechanical Turk worker withanamelikesmucker thinks that Amazon should place higher restrictions on requesters on the site. He thinks that there are still loopholes for them to abuse workers.

"Requesters should have to use a real, identifiable business/human name. Right now they don't," he said. "Anyone with a dollar on a Wal-Mart money card can make a requester account, put up 50 penny HITs, reject all of the submissions, keep the data they've collected, and do the same thing tomorrow," he told me.

While academics do make up a significant amount of the individual requesters on the site, the bulk of the actual work posted to MTurk tends to come from private businesses. While only 31 percent of requesters were companies, their tasks made up 83 percent of the work that was available on the site during Pew's study period.

A small group of five companies (Pew doesn't say which companies) made up the vast majority of work, accounting for 53 percent of the tasks posted. This indicates that some companies are incorporating Mechanical Turk into their regular business model, instead of just utilizing it for an ad hoc task.

Long term, some Mechanical Turk workers think that choosing to pay so little could affect the quality of work requesters receive.

"I think if requesters paid fairly and reasonably, and used the tools at their disposal to block bad workers from their hits, they would see an increase in data quality because people concentrate harder on the tasks and look over their work more carefully if they aren't hauling ass through the task trying to keep the pay rate above 6 bucks an hour," HydropnicFunBags said.

Motherboard has reached out to Amazon for comment but it did not return requests in time for publication.