Virtual Reality Without Awkward Body Sensors

For virtual reality worlds to be truly immersive, body tracking and haptic feedback—the virtual simulation of gesture and touch—will have to be enhanced. Whereas VR accessories like Leap Motion and Lucid Trips achieve this with varying degrees of success, another system aims to take the camera sensors off the user's body completely.

Imagine an entire room turned into a real-time virtual reality sensor. No walls painted to create a blue screen, and no motion capture suits to replicate one's body in virtual realms. Instead, a system of camera sensors that allow a user to move freely throughout nearly every inch of the space, with the sort of accuracy and flexibility that currently isn't possible with just a depth sensor fixed to a VR headset.

This virtual reality room exists, and could be used for everything from business and research to gaming and education. HeroVR, a collaboration between the Dusseldorf-based Agency for Virtual Reality (A4VR) and motion capture company The Captury, uses VR headsets and an array of 16 cameras placed strategically around a room to track a user's movements in real time. The idea is that the number of cameras can only increase the faithful replication of a user's real-to-virtual movement.

"This enables our customers to walk through virtual spaces while [they] see their virtual body reacting and interacting naturally and intuitively in VR like real life," A4VR explains on their YouTube preview. "There is no need to learn new interface schemes or any other things that are confusing or distracting from the experience."

The first demonstration of HeroVR came at FMX 2015 in Stuttgart, Germany, a conference on animation, games, effects and transmedia. This demo, as A4VR's Michael Albrecht told Motherboard, was a proof of concept.

As seen in the video, HeroVR places a visitor in a "futuristic training room" to train coordinative and motor skills. Each arm is equipped with a holographic shield to protect against hits. Users are able to freely move within the tracked area; which means they can walk, jump, duck and dodge attacks or block them with the shields.

The demo also placed an emphasis on three-dimensional sounds to help simulate the virtual space. As Albrecht said, the spatial 3D audio matches the user's head movement, enhancing the full body interactivity.

Right now, according to Albrecht, HeroVR will be limited to exhibitions. Users are able to interact with big virtual objects such as cars, industrial design, and architecture, created through computer-generated 3D animation.

They could also scale down to nanometer size in an animation and interact with molecules and materials; Albrecht said this would allow them to get a feel for physical interactions on a different scale, and could prove useful in scientific research.

And, as he noted, the tracking is not limited to one person: Small groups could experience the same VR world.

"It's perfect for any VR experience where you should inhabit that virtual space and where natural interaction with your body and high immersion is desired or the goal," Albrecht said. "In the future we will combine it with finger tracking for more improved interaction and optimizing the skeleton tracking further more."

In the near future, A4VR hopes to tweak the software to release a consumer version of its system, which could be used for gaming or other immersive virtual reality worlds.

Watch: In 2011, Motherboard headed to upstate New York to visit The Virtusphere, a 10-foot rotating virtual reality space used for research