Here's Why It's So Hard to Gauge the Game Industry's Health with Sales

The video game industry endured a bout of doom and gloom on Thursday after market research firm The NPD Group reported that overall sales were flat compared to 2014 in its 2015 annual report. That didn't sit well with the Entertainment Software Association, which puts on the annual Electronic Entertainment Expo (E3) and advocates for the games industry in general. NPD's claims are only right to an extent, ESA argues, as its "incomplete" data fail to account for significant digital shifts shaping the industry in recent years.

"Scores of millions of consumers purchase innovative content in myriad ways, including subscription services, digital downloads, and via their mobile devices," The ESA told Motherboard in a statement. "Gone are the days when the industry's growth and strength could be determined by retailers simply reporting packaged good sales."

This isn't exactly a surprise: industry pros have long known NPD usually doesn't include digital sales in its monthly reports (or even sales of used games, for that matter). It shapes its annual report to fit the needs of the retail industry rather than consumers, and its focus is on the United States alone. That "traditional model," as ESA calls it, might have served the industry fine in the past, but its brief statement suggests there's considerable harm in presenting such figures as accurate descriptors of the industry's health.

"Consumers, the creators and innovators of our industry, and the investment community deserve better," ESA wrote. "ESA will do its part by encouraging and motivating NPD to release the total consumer spend for 2015 that includes all aspects of this diverse industry."

It didn't take long for NPD to release a statement in return, in which it claims it's "fully committed to reporting on the total industry, which we have done for several years at the category level through our Games Market Dynamics service." It reasserted that its annual report focuses on physical retail data, and that the group has "been clear on the scope of these monthly reports."

"NPD's monthly retail/e-tail data release covers physical sales, which remains a very significant portion of total industry sales, but is not representative of total video game industry revenue," the statement read.

I contacted the ESA to learn why they released the statement if NPD does, in fact, report digital sales separately, but they have yet to respond. It's likely, though, that ESA knows how easy it is for onlookers to misinterpret the information as a sign of the industry's health. As things stand, it's a little like inferring that Taylor Swift's albums aren't selling well on iTunes because a relatively small number are bought at Walmart.

By its own admission, the NPD has been at least tracking consumer spending on both physical and digital purchases since 2010, with the latter covering everything from full games and downloadable content to mobile games and subscriptions. In fact, at the time, an NPD press release garnered considerable attention after it revealed that U.S. consumers had bought 11.2 million PC games online in the first six months of that year—a considerable jump over the 8.2 million physical games sold at retail.

In a 2013 interview with Games Industry International, however, ESA president Michael Gallagher called for a "standardized reporting mechanism that is seen as whole, and complete, and truthful." Not long after, NPD president David McQuillan announced in a Games Industry interview that NPD had formed a leader panel with nine "leading" publishers to "track digital [point-of-sale] sales of full game and add-on content downloads." In addition, he said NPD would work with the Electronic Entertainment Design and Research firm to track digital sales on a global level rather than North America alone.

"We couldn't agree more with ESA," McQuillan said at the time. "Full market transparency, including digital is incredibly important to the health of the industry."

It's 2016, though, and we're still seeing variations of the same argument. Yet the absence of the data isn't entirely NPD's fault; even back in 2012, Penny Arcade Report noted that a major flaw of NPD's methodology is that large digital distributors like Steam don't readily share their sales data. That's huge, especially since Forbes claimed that Steam might command as much as 70 percent of the digital distribution market way back in 2011. Back then, there were 30 million accounts on Steam; last year, though, Steam owner Valve announced there were now 125 million active accounts. And it gets more complicated.

"Generally speaking, there's an NDA clause in Steam contracts that says that confidential information like sales data can not be shared unless both parties agree to share it," an anonymous developer said in Penny Arcade Report's 2012 coverage. "So if the developer or publishers ask Valve and they say yes, they're free to share sales data."

That doesn't even touch on other digital distributors such as Apple, which keeps a similarly tight grip on the specifics of its App Store sales figures. Instead, we're usually stuck with sales reports from the game companies themselves or estimates based on observable data, such as SuperData's claim in November (via Games Industry) that the 1.87 million units of the blockbuster Fallout 4 were sold in the game's first three days of release alone.