Developers Use the Xbox Kinect to See Inside Your Head

Getting your head opened up in a non-figurative way—that’s to say, with a bone saw and scalpel—is kind of an all-or-nothing proposition. If you and your brain surgeon are going to take the trouble, you want to get it right.

Thanks to technology like CT scanning and MRIs, brain surgeons looking for tumors don’t have to go in there without a road map. Still, as any one who’s ever used a road map will tell you, a map is still just a map. Lining it up with reality requires a lot of back-and-forth comparison.

A new bit of technology presented at the Microsoft TechFest, however, may help eliminate the need for making those side-by-side comparisons, allowing doctors to virtually “see” inside your head in 3D without ever wielding a blade. And it uses a $110 piece of technology known to gamers all over: The Kinect, for Xbox 360.

Consumers can’t peer inside their fellow gamers’ heads using a Kinect fresh out of the box, of course. To devise the tool, developers used a Kinect Sensor, which works by “observing” your gestures through three lenses that triangulate your location, registering the space you occupy and the movements you make. As “Ben,” a researcher from Microsoft, explains in the video above, they first attached it to a run-of-the-mill, USB-powered touch screen and a standard monitor, allowing them to see and adjust what the Kinect was “seeing.”

The researchers then hacked the Kinect to fuse it with images from multiple 2D MRI scans. Because of the way the Kinect is able to effectively “grab” and register objects in front of it—usually a flailing body but, in this case, a fake skull—the developers rigged it to transpose MRI data in three-dimensions onto the real-world skull in front of them. The device can then be moved around the skull to see what’s inside of it from every perspective, giving doctors the ability to effectively see inside their patient’s heads.

Using typical MRI scans “for diagnosis and surgical planning,” Ben notes, “the problem is always, ‘How do I get this information, which is 2D and mounted on a wall, for instance, into the operating theater and basically in-situ on the object that I want to perform the surgery on?’

“Now,” he adds, “the physician will have the ability to walk around the operating room and look [inside] the patient’s head from different views.”

As is, the device probably isn't too practical. Ideally a doctor needs his/her hands free in order to better examine—or even begin operating—while looking at the 3D imagery. But the applied science is encouraging. One can easily imagine technology like this fused with something like Google Glass, to give doctors the ability to simply mount the technology to their eyes—just like X-ray specs.