Virtual Reality’s Locomotion Problem

By the end of the Game Developers Conference in San Francisco, I had spent more time in virtual reality than I did in real life. I was a quarterback in an arena full of cheering fans. I was a spy, sneaking my way around robot guards. I was a rockstar, shredding sick guitar solos in front of a screaming audience.

In retrospect, the strangest thing about a week of demoing virtual reality games—stranger than feeling the phantom weight of the headset's tether cable on the base of my skull and spine—is how still most of these experiences were.

There were exceptions, like the space dogfighting game Eve: Valkyrie, but most games allowed me to move very little, or not at all.

This is not a coincidence. If developers knew how to make a first person shooter in VR, where players can move around freely, that game would probably lead the pack of VR launch games, but they don't. By far, the issue VR developers are talking about most is this locomotion problem, or in other words, the fact that moving freely through virtual spaces, a fundamental appeal of video games for decades, makes most people sick in VR.

Assuming VR will catch on like these developers believe, how they solve this problem could change video games forever. But is that really what we want?

BABY STEPS

Sony, which tests and certifies every game before it's allowed to be published on the PlayStation 4, has created a "comfort sample" for the PlayStation VR, software that runs developers through a gauntlet of what they shouldn't do.

"Internally, we call it 'the bad demo,'" Sony senior staff engineer Chris Norden said.

It lets developers add latency, drop frames, change the field of vision, change the interpupillary distance (IPD), display shadows for objects only for one eye, and about 20 other parameters. These are just some of the variables that—if not perfectly tuned—can make people sick in VR.

The demo can be so nauseating, it comes with a "kill switch" button players can press to make it stop immediately. "This is the reason we're not providing the source code," Norden said. "We do a lot of things you should never ever do, and we don't want you to learn how."

It's in the interest of all parties to keep faulty reality away from users, and Sony, Valve, and Oculus all have the quality control systems in place in order to do that. Oculus has even devised a new "comfort" rating system, which divides its launch lineup of games into "comfortable," "moderate," and "intense" categories.

From an evolutionary perspective, we're sensitive to motion sickness because the vestibular system, which helps the body keep balance like a gyroscope, also acts as a defense mechanism. If I drank too much alcohol it would interfere with my vestibular system, making me feel like I was moving even though my body was still. The disconnect between what the body sees and feels is why people throw up when they feel like the room is spinning. Conveniently, throwing up also gets rid of the alcohol.

Moving in virtual reality while your body stays still causes the same kind of motion sickness. That's the locomotion problem. Exploring virtual spaces has been a fundamental element of the most popular video games for decades. From Grand Theft Auto to Assassin's Creed, a huge part of their appeal is that they allow players to move freely across cities, jungles, or entire galaxies.

This is not a problem with Northway Games' Fantastic Contraption, for example, because I could only move in the game by physically moving my body in the room.

Fantastic Contraption put me in on a floating, green platform under bright blue sky. It's a colorful, abstract space with no hard edges—a non-threatening, Fisher-Price reality that's opposite to the most popular, grim video games out today. The game asked me to build a machine that will transport a pink ball into a pink block. To build, I simply pulled sticks and wheels from my green cat companion using my two hand-tracking controllers, and connected them together at the joints. If I needed a stick to be longer, I pulled it from both ends to extend it. If I needed to re-adjust the alignment of the wheels, I got down on my knees and moved them with my hands. It's a game you play with your entire body, walking around the room, picking up virtual objects and putting them together.

A surprising number of games at GDC have taken this approach. Rather than try to solve the locomotion problem, they're avoiding it altogether by limiting gameplay to a confined space you can physically walk around.

In Schell Games' I Expect You to Die, for example, the player is a secret agent who must survive a series of deadly situations by solving puzzles in small spaces like an office or car.

"You want to avoid virtual camera motion," CEO of Schell Games and professor at Carnegie Mellon's Entertainment Technology Center Jesse Schell said. "I know that's unpopular, but what we're finding more and more is that traditional video games are about explorations of space, while virtual reality games are about manipulation of objects."

Job Simulator, which is emerging as another early VR hit, is an object manipulation masterpiece. It made me the Buster Keaton-like star in a series of regular jobs. I was a paper pusher in a cubicle and a clerk in a convenience store, but it's all slapstick. The first thing you want to do in VR with motion controls is pick up whatever isn't nailed down and throw it across the room, and Job Simulator provides the wish fulfilment of throwing coffee mugs at your coworkers and punching every juicy button in the room with your fist.

The other popular solution to the locomotion problem is the cockpit. Running in VR while your body is still causes motion sickness, but we're all more familiar with the sensation of watching the world pass by while sitting in a vehicle. That makes porting space games like Elite: Dangerous to VR much easier, but paradoxically not that much different than their regular versions. It's the same space game, but you can move your head to look around.

It's easy for players and critics to get lost in the minutiae of video game design, to mistake incremental tweaks between games as actual change ("this Call of Duty has jetpacks!"), but in reality most video games have been iterating on the same ideas for about two decades. The transition to 3D, polygonal graphics, and online, multiplayer games in the late 90s were the last time game developers had truly new frontiers to explore.

The things I did in Fantastic Contraption felt like they could be the beginning of an equally significant change. This is what's exciting about VR: Developers have a chance to create experiences that are fundamentally new and unique.

But how many games confined to small spaces will we want to play before we want to go back to exploring entire continents? Will we never see a first person shooter where we're free to explore like we are with traditional video games?

There used to be a time when people thought there's no good way to play a first person shooter on consoles either, that the only way to play them was on a PC with a mouse and keyboard, but then Halo happened.

"Solving the locomotion problem is something we wanted to do," Jaime Griesemer said. He was a designer at Bungie who worked on the controls for the first Halo, and is currently the creative director at developer Highwire Games, which is working on its first VR game, Golem. "You have to be able to explore to play a game, especially a 3D game like we like to make. People were telling us it's impossible, a lot of people have given up already, and VR just started."

Griesemer's solution to the locomotion problem uses VR's head tracking. If I wanted to move forward, I leaned forward. If I wanted to move back, I leaned back.

You'd think that if someone could solve the locomotion problem, it'd be the designer who's had so much to do with the way players locomote through games today, but I've played Golem, and while it may be a fine game when it's released, I don't think that this is how we'll play VR games.

In Golem, I played as a paralyzed child with the ability to embody and control inanimate beings. I started out as a little doll on a shelf, scurrying around a room like a mouse until I found a way out through the floorboards under the bed, and finished the demo as a giant rock monster with a sword, fighting other giant rock monsters by swinging the motion controller. I had to sit up straight in order for any of this to work, hold out my arm, and work my core a little to lean in the desired direction, none of which was easy because my core is like a bag of melted marshmellows.

If Doom makes you feel like a badass because it lets you destroy legions of demons with the tips of your fingers, and Fantastic Contraption feels amazing because you're solving problems with your entire body, Golem finds a middle ground I wasn't looking for. It is the wrong input to output ratio.

The best solution to the locomotion problem I played was Budget Cuts. It's a first person stealth game where I avoided robots or took them down with throwing knives. To get around a level, I shot a ball of light where I wanted to go, then pushed another button to teleport there instantly, kind of like Valve's first person puzzle game Portal. I could physically walk around in any given space once I was there, but I could also easily get around infinite virtual space by teleporting myself across rooms and hallways.

Budget Cuts was made in six months by Neat Corporation, a team of only two full-time developers from Sweden.

"We started doing two weeks of prototyping when we got the Vive developer kit," Neat Corporation co-founder Joachim Holmér said. "We're all sensitive to motion sickness, so we wanted everyone to be able to play this. Teleportation is a good solution because there's no continued acceleration."

It's entirely possible that these two developers you never heard of have solved the locomotion problem, but that doesn't mean teams of this size will continue defining VR games.

AAA Looms

None of the game developers I talked to at GDC that were working on VR had a team that was bigger than 20 people. Even Adr1ft, which looks like the first big budget AAA game for virtual reality, was made by a core team of six people.

[This profile of Northway Games perfectly captures the optimistic attitude about virtual reality right now. Credit: Breakwater Studios Ltd.]

I'm happy to report that after talking to many of them and seeing them speak at GDC, these developers seem like well-intentioned people who are trying not to repeat the mistakes of the past. They're trying to make video games more inclusive, safe, and welcoming than they are today.

"There's a shared responsibility in the content we create," Kimberly Voll, a cognitive scientist who worked Fantastic Contraption, told the audience during her talk at GDC. "We have to think about trust and safe spaces. We need to think about the research that still needs to be done about long term effects to kids, and so on. If we can't do it we need to talk to the people who can. This is an exciting time and we want to keep the momentum, but there's due diligence here as well."

The big video game publishers are noticeably absent from the VR race at the moment, with a few exceptions. Ubisoft, which has a history of betting early on new platforms, announced it has dedicated small teams for two relatively small VR games: a party game, Werewolves Within, and a flying game, Eagle Flight.

Insomniac Games, a AAA studio which worked on the Ratchet & Clank series and more recently on Sunset Overdrive, has also dedicated a small team to Edge of Nowhere, an action adventure game in the style of Uncharted set in a Lovecraftian Antarctica.

But mostly, major publishers aren't ready to bring their popular franchises and marketing power to VR.

"You're going to see big studios launch on virtual reality when the market's established and people are buying things and they can make a lot of money," said Harley Baldwin, vice president of design at I Expect You to Die developer Schell Games.

For now, it makes more sense for smaller developers to solve the locomotion problem.

"Being a smaller team and being very agile works in our favor because we're able to make hard choices and pivot our whole team if we need to," Baldwin said. "It's hard to imagine what would happen in a much larger company if they tested new levels and were like 'oh, they're not very good.'"

That kind of bet is too risky and expensive for giant, AAA teams.

The question, then, is how long these small, idealistic teams can keep experimenting with VR. Supposedly, there's a fortune to be made by getting in on the ground floor of a new platform. But Wanda Meloni, executive director at the Open Gaming Alliance said that consulting firms like Digi-Capital, which previously estimated the virtual reality market will hit $30 billion in revenue by 2020, might be wrong.

"A lot of analysts are now cutting back on their estimates, and I think we're going to see more of that," Meloni told a sparsely attended, off-site panel organized by research firm JPR. "The hardware is awesome, but we have to get the software to a place where we have a stable ecosystem. The issue is that we have these developers that need our support. The only way right now that they're actually getting funded is through investment and we need to find a way so they can actually start making money."

This is VR's chicken and egg content problem: AAA game makers are not going to invest million of dollars in building games for a VR audience that doesn't exist yet, but it's possible that this mass audience is waiting for AAA VR efforts before they buy a headset. To be clear, I've seen a lot of cool shit in VR, but nothing there was a system seller or killer app.

If there's any money to be made in VR, it's unlikely that AAA developers will sit on the sidelines for long. If I were a mega publisher like Activision or Electronic Arts, I'd sit back and see which of these small developers comes up with the best solution to the locomotion problem, and which game resonates with early adopters the most. I'd then acquire that team, and put the full force of AAA game production behind it. (Either that… or I'd skip the acquisition, steal the idea, and put guns in it.) What does a virtual reality game look like with a multi-million dollar budget and a team of 100 people?

I think that the people who are most excited about virtual reality—people who are already playing a lot of video games on consoles and high-end PCs—are hoping that it will make the type of games they already love better and more immersive. Most game developers say they don't want to play, let alone make, something like a first person shooter in VR. But I suspect that's what players want, and if one video game figures out how to do it, others will follow.

GUILTY PLEASURES

I played one VR game at GDC that was a lot like a traditional first-person shooter. It's called called Bullet Train, a clever title, since it included both bullets and trains. It's only a proof of concept, but I moved in it much like I did in Budget Cuts. An important difference is that instead of teleporting wherever I wanted like I could in Budget Cuts, in Bullet Train I could instantly teleport only to predetermined spots around the level by pointing at them. It's a less interesting way to move the requires less thinking.

The demo starts when my train arrives in a train station. The doors slide open, and I'm immediately attacked by a group of Bad Dudes in heavy body armor. I teleport across the tracks, behind them, pick up an AK-47, and mow them down. By then, more Bad Dudes start flowing into the station. I teleport up the stairs, grab two pistols, one in each motion controlled hand, and start firing in slow motion like the star of a John Woo film. I can slow down time, grab bullets out of the air, and throw them back at the enemies. When I teleport around a corner and nail one of them with a shotgun blast to the belly, he goes flying across the station.

I murder dozens of these idiots, as I have in so many video games before, and I'm smiling the entire time. I enjoy shooting people in video games. So sue me. I'm not proud of it, but if I was given the choice between playing all of Fantastic Contraption or whatever a full version of Bullet Train is, I'd probably choose the latter.

I know that virtual reality can make video game so much more than Bullet Train, but I think there's a big gap between the idealistic conversations I heard between game developers at GDC and the audience that's actually willing to put down the money for VR.

I know because I am that audience, and deep down in my dumb gamer brain, ravaged by decades of developers catering to my basest urges, VR is not going to be a real platform for video games until I'm putting bullets in things. I want to escape. I want to turn off my brain and play something that makes me forget and gives me the illusion of order, control, and accomplishment. I want something familiar.

"This is a really incredible time," Voll told the the audience of developers at GDC. "We here together are establishing the language of VR and the rules of VR. We're the stuff that 10 years from now, if textbooks are still a thing, they'll be writing about what we're figuring out today. Welcome to the new frontier."

The developers responded with a passionate round of applause. I'm inspired too, but part of me worries they're overestimating their audience.