You Can Scan Your Face Into 'NBA 2K17' With Your Phone

NBA 2K17 is the latest version of 2K Games' sports simulator, and it comes with an innovation that is sure to make many players very happy. For the first time in franchise history, players can scan their heads directly into the game via their mobile devices.

Yes, NBA 2K15 and NBA 2K16 both had scanning options, but a player was limited to using either the Kinect camera on the Xbox One or the PlayStation camera on the PlayStation 4. Mobile devices are the new equalizer—now, everybody who purchases NBA 2K17 can enjoy the game to to the same extent as everybody else.

Personally, this is a relief. For me, the most frustrating part of any sports video game is the first 40 minutes. Why? Because that's how long it takes for me to work through whatever Create-A-Player mode the developers designed. I fiddle with sliding knobs that determine forehead size or nose bridge dimensions, and I ask myself: "How high are my cheekbones, really?" "How much upper lip space is there between the bottom of my nose on the top of my mouth?" I have a better chin than that, right?"

I'm ethnically Chinese. And there is a major shortage of Asian features in sports titles, particularly in the variance of eyes. Most ethnic East Asians have a characteristic epicanthic fold, which covers the corner of the upper eyelids and gives Asian eyes their distinctive, narrowed look. It's rare -- extremely rare -- to find a game which provides this option, notwithstanding the customization to create varied shapes.

There's also the instinctive, ego-preserving inclination to design the best version of oneself. But once I swallow my pride, I am able to create a player that bears a passing resemblance to me; it's close enough that I can bear looking at him for the next several months. But he's definitely a bit off. And he certainly doesn't look very Asian.

But facial scans, at least in theory, promise to change that, because of how the technology works. It captures the face's topography as a collection of reference points—anywhere from 5,000 to 15,000 points in proper, even lighting. So theoretically, everything from eye shape to nose shape to mouth shape ought to be accurately depicted.

"It's a photo based technique," says Tim Meekins, a gaming software engineer for Visual Concepts, the company responsible for this technology. "The software looks at 10-15 images, and based on the angle of those images, it can triangulate a depth [for the person's features]. It shrink wraps your head around those reference points. You can then project textures over that, and that's what makes it look like you."

"Think of it as a reverse panorama," adds Dan Indra, a game designer for Visual Concepts. "As you're turning your head, the program captures the panorama around your head."

Visual Concepts has been developing this facial scanning technology in-house for three years. The transition from console camera to mobile device came as welcome news to both men, who envisioned more opportunities than obstacles as a result of the switch.

"As nice as it was to scan your face directly into the camera on the console, you had to be directly in front of your console to do it," says Indra. "And that didn't necessarily give players the best lighting."

"The regular cameras that come with the consoles are simple LED cams," adds Meekins. "They aren't like modern cell phones, like the iPhone, which have double or triple the resolution."

"These front facing cameras—these selfie cameras—are also made to do exactly this," continues Indra. "And that's not necessarily what the console cameras were designed for. The Kinect in particular has an incredible camera, but it wants to see the entire room. It wants to see your whole body. It doesn't like when you try to stand 10 inches away from it. And so, you have to stand back from it, which means that you lose a lot of resolution."

The mobile process that I went through was more straightforward. After installing the game, I downloaded My NBA 2K17, the mobile partner app that pairs with the game. The app asks you to take 13 photos, each with your head tilted slightly to the left or slightly to the right—a 60-degree arc in total.

The results can speak for themselves. Here's a picture of me in profile:

And here is my 2K17 doppelganger, before I had the chance to add his hair or add his eyebrows.

This was considered an "average" scan, according to the game. The lighting in my living room is a bit uneven, and 2K recommends taking the photos in even, bright light to get the best possible scan. Still, it's quite impressive. That is me, for better or for worse; the camera is a lot more honest than I would be with myself. And if I really wanted to, I could sculpt my features further to smooth out any irregularities. Again, here is a picture of me:

And here is an in-game screenshot of my avatar. This time, I've added in hairstyle, facial hair, eyebrows, and skin texture to give myself a more weathered, realistic look. It's as close as I've ever managed to make an in-game avatar look like me.

Indra and Meekins expressed hope that in the future, the program will get better at rendering unique, ethnic characteristics. The program has to be trained and taught to recognize human features and fit to them; the more faces there are in the database, the more variability the software can account for and readily associate with a type.

Read More: Facebook uses a similar technique to teach its software to recognize common objects and locales.

"The more heads we can train the system on, the more [the program] will be able to bend, contort, and form in more ways," says Indra.

Currently, out of the hundreds of heads that are currently in the database, the developers have scanned nearly every active player in the NBA and most of the coaches. By uploading additional heads into the database, the team hopes that that the program will evolve to be more flexible. But it is uncharted territory; even the developers aren't positive how far they can push the customization.

"We're not entirely sure," says Indra. "We're learning, and we're on the cutting edge. There's no precedent, and there's very few people who are doing what we're doing."

"The way the code works, it should be able to solve any head," says Meekins. "We're looking into throwing in some random faces of people in the office and on the street to give the program some extra training data. We've been doing that already, and we want to expand and do more of that for next year."

Get six of our favorite Motherboard stories every day by signing up for our newsletter.