The Video Game Controller that Can Read Your Emotions

The idea is to track how you feel about a game and throw more zombies at you if you're bored.

Apr 8 2014, 12:15pm
Image: Youtube/Stanford University

Corey McCall has reverse engineered what's happening in your brain to make a video game controller that can read your mind—or at least your emotional state. He eventually wants to make video games that adjust themselves based on what you're feeling while playing them. 

The Stanford doctoral candidate managed to transform a regular Xbox 360 controller into an emotion-reading device by ripping the back panel off and attaching a box full of sensors. He also added some metal pads that measure heart rate, blood flow, breathing rate, and how deep each breath is. On top of that, he stuck several accelerometers inside to measure how frenzied people get while button mashing.

The science behind it isn’t anything new: The brain’s autonomic nervous system—the emotional part of the brain, which changes upon excitement, boredom happiness and sadness—influences your heart rate, respiration rate, temperature, perspiration and “other key bodily processes.” Measuring those biometric signs with all the sensors lets McCall record how a player's feeling about the game they’re playing.

Video: Youtube/Stanford University

Ultimately, the goal of collecting that kind of data in real-time is to add a further layer of interactivity into video games—so a game could adjust, for example, the number of brain-eating undead hordes attacking a player depending on whether or not they’re bored.

Since there isn’t any software out there for Xbox that can use McCall’s toy, Stanford researchers had to build their own custom game—some kind of futuristic-looking racing game with Tron-like graphics. At this point, they’re just collecting data about how people’s physiology responds, and the player’s level of “mental engagement” while playing video games; they aren’t actually changing a game in real time. But eventually that’s exactly what McCall wants to do, and get it out to a wider audience.

So far getting brain-reading technology out to a larger video gaming audience is something few in the industry have been able to figure out. It’s not for lack of trying. Back in 2013, Sony was playing around with the idea of adding a sweat sensor (yum!) to its Playstation 4 controllers, but eventually dropped it for reasons that aren’t entirely clear.

Valve, too, has commissioned studies that aim to track how much gamers sweat while they’re playing various first person shooters, and also to use eye-tracking to actually control a game.

And then, of course, there’s the mind reading video game helmet we wrote about that’s supposed to control video games with your brain. While that didn’t really work out for the company, Emotiv and others have managed to find some pretty interesting scientific uses for the product—like studying orgasms.

 Time will tell whether or not McCall’s emotion-sensing Xbox controller will suffer the same fate as other biometric gaming tech, but the researcher has said there's been a substantial amount of positive feedback so far.