Creepily Realistic Avatars Are Giving 'Second Life' a Second Life

"It'll scare the hell out of you."

Apr 25 2014, 5:55pm
Image: High Fidelity

Creating a virtual world isn't something you can fudge through half-assed. Building a digital universe people actually want to live in is extremely difficult, and a VR-powered metaverse clone remains a chronically emerging trend that won't take off until it's just about perfect. That's why the Oculus Rift stayed a niche (albeit beloved) novelty product for so long, and it's why most people haven't lived an entire "second life" in the digital dimension.

It's also presumably why it's taken nearly 20 years for Second Life founder Philip Rosedale to develop the next-gen version of his world. The new company is called High Fidelity, and though it's been hiding in stealth mode, quietly raising millions in VC cash for over a year, Rosedale's now starting to offer a sneak peak at what he's been working on.

New Scientist's Samantha Murphy was one of the first people to give High Fidelity's VR system a test run, which she described in an article yesterday. Rosedale also showcased the project at the Silicon Valley Virtual Reality Meetup last month. You can watch it below; his presentation starts around the 42-minute mark. 

As someone with only a marginal interest in gaming, but a lot of interest in how virtual reality can upend society, I can say that this is a very cool project. True to its name, High Fidelity is focusing on how to simulate reality with eerie accuracy, specifically by making a richer social experience—something that was lacking in Second Life 1.0.

Unlike many other VR apps, HiFi's focusing on making the virtual world a social space, where you can talk and interact with people just like IRL. The secret sauce is body language. The High Fidelity avatars mimic the user's facial expressions, unique mannerisms, and gestures as the user moves and talks in real time.

"The robot seems to be smiling at me, tilting his head, blinking and opening its mouth as if to speak. It dawns on me that it is mimicking my facial expressions by watching them via webcam. The robot was me," wrote Murphy.

Real real-time is the key. The next-gen system reduces latency, the scourge of VR tech, down to 100 milliseconds, said Rosedale. That's twice as fast as Skype and four times faster than talking on a cell phone.

It's like looking in the mirror, Rosedale explained at the conference, only you're looking at an animated version of yourself. They tried using photorealistic avatars instead, but that combined with the low-latency, expressive features and true-to-life voice turned out to be way too creepy. "It'll scare the hell out of you," he said. 

For now, achieving the effect means rigging users up with all sorts of high-tech equipment. Just as high-speed internet enabled the birth of Second Life, its rebirth will be built on the backs of gesture tracking, motion capture, handheld controllers and next-gen, low-latency headsets like the Rift. Here's one version of the set up, as described on the High Fidelity blog:

A MacBook Pro with PrimeSense depth cameras clipped to the top of their screens (we 3D printed the clip), the Faceshift SDK extracting head position and facial features, and our Interface software processing and streaming the data to control the avatar’s face and body. Ryan’s hand is detected by the Leap Motion controller. The end-to-end latency is about 100 milliseconds. For our headphone and microphone, we usually use this Sennheiser headset.

It'll also be built on the backs of the millions of electronic devices consumers now own. Rosedale’s other plan to revolutionize the VR space jives with the currently popular effort to decentralize the web by divorcing internet traffic from centralized, corporate-owned servers. High Fidelity plans to leverage the space on people's laptops and smartphones to build a network of mini-servers, which allows for much faster connection speeds.

I watched a couple videos demonstrating HiFi's avatar interactions, and it's certainly impressive, as far as avatars go. The expressive faces looked incredibly human-like, which adds a whole other dimension to the interaction by invoking emotion. Past studies on the psychology of VR have shown that humans are inclined to identify and empathize with our digital doppelgangers, and the more humanoid, the more the divide between the physical world and digiverse breaks down.

That said, there's still a long way to go before the looming VR renaissance, despite a growing chorus of buzz around its arrival. The decline of Second Life left a super-nerdy stink on virtual reality that's been hard to destigmatize, though Oculus getting snatched up by Facebook is a sign that VR may have a mainstream chance yet.

If people can break past the mental barrier of watching an animated robotic version of themselves in cyberspace chill with other fabricated avatars, perfecting virtual communication could open up virtual reality way beyond gaming.

It could become a way to hang out with your friends—with visuals, movements, and a choose-your-own environment to boot, which makes our current version of online social networking, "liking" status messages and uploading photos, seem clunky and antiquated. It could also be a way to conduct business; High Fidelity itself holds team meetings in the metaverse. It could rethink trends like online education and virtual therapy, if the students or patients step into a virtual office and can see what their teacher or doctor is doing and feeling during the discussion.

Potentially, it could redefine our entire, analog concept of place and time. Rosedale thinks it’s going to be as big as the internet—society’s next big leap forward. At HiFi the motto is, “If it doesn’t hurt to think about it, we’re not going to try it.” Still, that's a big "if."