Finally, Virtual Reality Let Me Fly Like a Bird

I never thought I'd be able to fly. But there I was, facedown on a padded, cross-shaped platform, wearing an Oculus Rift headset and a pair of headphones. I flapped my wings, until suddenly I hung over San Francisco. I could see Market Street, far below.

It was just the beginning of my time with  Birdly, an elaborate, full-body virtual reality device that simulates the experience of soaring like a bird. Birdly's creators have taken their simulation on the road, starting in Swissnex, an annex of the Consulate General of Switzerland in San Francisco, where I recently had the chance to give the thing a go.

Turns out, it's a lot of fun. But Birdly's real value is that the simulator is addressing a problem that Oculus-style and other virtual reality game developers have been avoiding so far. For all the progress that next-gen VR technologies have made just in terms of image quality and low latency (it won't make you sick), a parallel problem, a kind of uncanny valley of presence, has seemingly gone ignored.

The more realistic the virtual world  looks, the more striking it is to have what the rest of your senses feel not match up. You're looking at alien spaceships firing lasers in Mars' orbit (or "experiencing" porn, or driving a military tank), but you're still touching and smelling the real world, your body slouched in a coffee-stained office chair.

That's where Birdly comes in. The simulator, created by a team led by Max Rheiner at Zurich University's Interaction Design Program with funding from the BirdLife wildlife organization, takes on this issue by creating an input and feedback device that engages the entire body, which I found to be a pretty powerful experience.

The two earlier Birdly prototypes that BirdLife kept rendered the Swiss countryside. In  that version of simulator, depending on what you're flying over the contraption taps users' sense of smell, dispensing controlled amounts of clean air, native flora, and other natural smells distinct to the Swiss countryside.

For the San Francisco demonstration, Rheiner and his team incorporated detailed 3D map data from PLW Modelworks to render the city, swapping out pleasant Swiss-countryside aromas for characteristically Tenderloin scents, like hot garbage and urine.

So with feet elevated, I hung my head over the edge. I hooked my hands under two handles, allowing me to raise and lower the arm rests. This is how I "flapped" my "wings."

I manipulated my wingtips with a subtle twist of my wrist, such that I cut down through the air, taking an abrupt nosedive at the streets below. As I tipped forward, so too did the electric motors beneath the padded platform slant forward, matching the angle of my body in the real world. At one point, I felt like I faced downward at a sheer 90-degree angle. Fabian Troxler, who helped create the contraption, told me that a few of the initial Birdly trial flights ended in face plants.

A fan installed in the front of the device also blew air to match the speed of descent, giving a realistic whip of wind to my face. San Francisco is rendered in crude 3D, but the sensation of plummeting toward Market Street from several hundred feet up in the air was nevertheless so real a knot formed in the a pit of my stomach. At the last moment, I twisted my wrists the other way, pulled up, and flapped like hell to my wings to gain altitude. The greater the wind resistance, the harder it is to flap.

A minute later, the real world fades away, and I'm in full control. I'm just a pretty little bird circling the Transamerica building, swooping in between skyscrapers, soaring leisurely above the city.

Birdly isn't entirely alone. Virtuix, a consumer-facing VR gaming company, has also caught on to the problems inherent to this so-called uncanny valley of presence, and is offering solutions like the  Omni, a $500 natural motion interface that looks like something between a treadmill and a vibrating belt weight-loss machine. It allows you to literally run around in a first-person shooter.

Besides managing to make the Oculus Rift look even dorkier, the Omni is also only a half measure. You have to run in the real world in order to run in the virtual world, but feedback is still minimal.

"Tactile feedback is hard to get right, and hard to make affordable for consumers," Virtuix CEO Jan Goetgeluk told me. "With every additional VR technology, the return insofar as immersion diminishes. An HMD [head-mounted display] provides arguably the highest return, since VR does not exist without the HMD component. Locomotion provides an additional, significant increase in immersion. Tactile feedback is currently not a must, as its added feeling of immersion does not make up for its current high cost."

There's also YEI Technology, which creates sensors that can track your body's movements precisely in real-time, offering a different kind of full body input method in the PrioVR. Essentially, it's a suit of sensors you can wear to capture every gesture and translate it on-screen.

"Our bodies are what allow us to connect with and interact with reality in a truly natural and intimate way," YEI Technology Chief of Research and Development Paul Yost told me. "As such, a virtual avatar body that moves with you and feels natural is required to achieve a true suspension of disbelief. It is this that can allow VR to move from novelty to a true and compelling virtual experience."

Yost also agrees that the ultimate VR solution is clearly one where all senses and sensory feedback from the real-world are reproduced in the virtual world. "This will happen eventually, but right now, we're still working towards getting all the major pieces working so that the technology can allow a suspension of disbelief that, in turn, allows for a compelling and natural-feeling experience."

Birdly is an early taste of what a more convincing solution might look like. Rheiner said it was inspired by humanity's shared dream of flying, which is entirely different from our actual experience of flight or traditional flight simulators.

"When you fly with a helicopter you're so mobile, you can land on roofs, so it's the ultimate freedom, but it's a fraud in the end," Rheiner said.

He told me about a friend of his who's in the process getting a pilot's license. This friend initially thought that by acquiring a license to fly "he would gain freedom" but in the end was apparently held back by a slew of checkups and security measures, with his instructor telling him not to listen to his instinct, but rather to the devices.

"For us it was the opposite," Rheiner explained. "We really want to have you let your senses go and just enjoy the feeling of flying."

It's far from perfect. As I flew, I could still feel the pressure of my weight against the Birdly platform, especially the bike seat-like protrusion on my crotch. When I got going real fast and took sharp turns, I could also feel the electric motors jerk beneath the rig as they struggled to keep up.

Even still, the input I give Birdly—and the feedback it spat out—was comprehensive enough to sell the whole thing. I didn't realize how convincing it was until I finally took off the headset. The transition from flying freely in the air to the grounded reality of the Swissnex gallery was jarring, and I have to say a little depressing.

Birdly works because Rheiner doesn't come from a strict engineering or game design background. He got his start in electronics, switched over to software development, and eventually found himself in art school with a Media Arts major.

You can pick up on the art school background immediately, because Rheiner doesn't talk about investors and commercial potential. He talks about dreamscapes, and the capital-B Body.

"I teach at the university about interaction, so my main research is to find out what other kind of senses you can use, because at the moment we have audio and visual, we use the hands, but that's it mostly," Rheiner told me. "Of course it makes sense, because we're very efficient with [our hands]. But certain kinds of information or experiences you want to transmit, you can't just do it over the eyes," he added. "It's just limited. That's why I really look around to find different kinds of ways to incorporate the whole body."

Rheiner told me he wants to go a step further, and simulate different types of dreams. To that end, he envisions a new contraption that could approximate the feeling of zero gravity.

"For now we just wanted to prove how far we could go and then we also would like to change the topic," he said. Dreamscapes would allow users to "fly through sound, and shoot sound," which then would be heard as feedback, Rheiner added. "You don't actually know what you do, but it feels good. [We] would like to simulate this kind of stuff."

It's unlikely that experiences like Birdly will be found anywhere outside art galleries or museums any time soon. But so far it's more interesting than anything the traditional games industry is doing with the technology. If that industry were smart, it would turn to people like Rheiner for a little inspiration.

After using Birdly I see the value in his pursuit. If we want a convincing virtual reality we can't rely on sight alone. We need to rethink and greatly expand all forms of input and feedback.

@emanuelmaiberg