Can a Medical Device Cure Virtual Reality Sickness?

​

Other than proving that I could have been a beautiful dancer, the picture above shows MindLeap, a new virtual and augmented reality headset from Swiss company MindMaze that aims to rehabilitate stroke survivors and amputees.

As a nice byproduct, it may also help solve one of the biggest problems in virtual reality gaming.

At the Game Developers Conference in San Francisco, the device's inventor and MindMaze founder Tej Tadi showed me a video of a stroke patient who had lost mobility in her left arm.

The patient watches a silhouette of herself on a screen, follows instructions on how to move her arm as best she can. A motion capturing camera similar to Microsoft's Kinect tells her how well she's doing. By watching herself and trying to move the affected arm, she more accurately targets the penumbra, the area in the brain surrounding the site of the stroke, which will help her recover functionality faster.

"What science has shown is that if you see me picking up this remote, you're still activating the same areas of your brain as I am, just by observing," Tadi told me. "That has been proven and we have shown it too with our own clinical studies."

So why is a health care device at the Game Developers Conference? Tadi believes the same technology can be used to solve the latency problem with virtual reality video games.

Nothing breaks Oculus Rift's illusion faster than latency, the time delay between when you move your head and when the simulation responds to that movement. Even a hundred millisecond of latency will cause virtual reality sickness, which VR games must overcome before they become mass consumer products.

The brain activity associated with moving your arm, for example, happens 500 milliseconds before you even move it, so MindLeap doesn't only need to catch up with your physical movements, it needs to know what you're going to do before you do it.

That's why electrodes are built into the headset. Tadi explained that before you're trying to move, there's what's called a desynchronization event in the brain. Different desynchronization events are associated with different brain activities, so MindLeap can identify them and figure out that you're going to move your arm as soon as that brain activity starts.

The electrodes and mind-reading algorithms that MindLeap uses to know you're going to move your arm before you move it can also be used to track other movements, allowing the device and the simulation to stay ahead of your actions and prepare accordingly instead of constantly catching up. If the virtual reality can stay ahead of your movements by reading your brain waves, there's no latency and no sickness.

Maybe you're thinking that this sounds like the kind of pie-in-the-sky, aspirational TEDx talk that is all very nice and good but ultimately a little divorced from reality, and yeah, here is a video of Tadi g​iving a pie-in-the-sky, aspirational TEDx talk.

I'll add to that skepticism that my experience with the actual prototype was not as impressive as it sounded. It's a functioning virtual and augmented reality device, but the image quality was still poor, and since the model I tried didn't have the built-in electrodes, the latency was even worse than the Oculus, Samsung, and Vive headsets that were also at the show.

Since latency is a huge problem that the best and brightest at Oculus and its competitors are spending so much money to solve, I also asked if Tadi if any of them have stopped by to take a look at his supposed solution. So far, they hadn't.

However, this isn't to doubt the idea. Tadi's research has been published and peer reviewed, and MindMaze's devices that don't use virtual reality are already in hospitals. It's currently in the process of being approved by the FDA and Europe's equivalent, CE, in hopes of expanding to 700 treatment centers in the United States and another 600 in Europe. Hospitals are already pre-ordering the device.

It remains to be seen whether MindLeap can help with gaming's virtual reality sickness, but it can help patients.