The Cyborg Who Can 'Hear' What Other People Are Looking At

Being recognised as one of the world’s first cyborgs is pretty cool, but now that cybernetics are getting more common and everyone’s starting to stick technology on their bodies through wearables or DIY implants, it takes constant innovation to stay ahead of the curve.

So when I heard Neil Harbisson had a few updates on his unique antenna-like modification, I was keen to see what could push the boundaries of the human body more than giving yourself a new sense. The answer? Sharing in other people’s senses. He’s modified his head-mounted eyeborg device so he can get sensory information from the other side of the world beamed directly into his skull.

He unveiled his latest upgrades in a talk at London’s Central Saint Martins College Monday night, where he’s currently working as a practitioner in residence to encourage students to think about a post-human (read: cyborg) future. In front of an intimate audience, he experienced “colour-to-skull communication” for the first time.

To recap a little, Harbisson is a Barcelona-based artist who was born with achromatopsia, which meant he was only able to see in black and white. So he invented the eyeborg, an antenna-like device that essentially lets him hear colours. A chip transposes the colour spectrum onto a musical scale, with red giving the lowest note and violet the highest (in fact, he’s expanded his sense beyond the human visual range to include infrared and ultraviolet either side too).

He constantly hears the notes of the colours around him play via bone conduction against his skull, so that a trip to the supermarket is an aural expedition. “It’s like a night club, there’s electronic music in every aisle,” he said, then added the disclaimer, “Milk is silent.”

Harbisson identifies wholeheartedly as a cyborg, hears colours in his dreams, and even convinced the British government to let him include his antenna in his passport photo—an achievement that arguably makes him the first “official” cyborg.

And so his first revelation of the evening was the culmination of a natural progression: he revealed for the first time that he has now had the eyeborg surgically implanted into the back of his skull.

Unsurprisingly, it took a lot of effort to find a doctor willing to go ahead with the procedure, but he finally had the operation in Barcelona last December. He showed photographs of the surgeons drilling into his head as he sat with his chin to his chest. The antenna is embedded in the occipital bone at the back of his head, with a separate hole for audio input—essentially, a jack drilled into his skull that transmits sound into his head through bone conduction.

At the end of the antenna, a modified camera detects both hue and saturation (more vibrant colours make a louder noise), and the whole setup is controlled by a chip. In a phone call, Harbisson talked me through the surgery. They had to remove a patch of his hair permanently to reduce infection, and reduce the thickness of the skin. “Then they opened the skin and they drilled—it was just drilling different holes for the antenna and the audio entry,” he said. It took around eight weeks to heal.

For Harbisson, the new antenna understandably feels even more a part of his body than before. There’s no pressure against the back of his head, the sound quality is better, and it feels like a body part. “If you touch the camera or the antenna it’s like touching a tooth or a nail—I feel it, basically, which is weird, because I didn’t feel that before,” he said.

The technology in the latest version of the device also included a very juicy little addition: thanks to a Bluetooth connection and a custom developed app, Harbisson can now hear colours that other people are seeing.

He demonstrated this new capability—which he hadn’t tried before—by hooking up to friends in Barcelona and New York (a third connection in Melbourne appeared to have overslept). They called Harbisson through the Eyeborg app, then used their smartphone cameras to look at different coloured objects. Harbisson could then hear the hues directly in his head.

He also experimented with getting one of the students present to point their viewfinder at different objects on a table and stream what they were seeing to his antenna. Going on the sound, he could correctly identify objects like his blue travel card, multicoloured tie, and burgundy passport.

I asked Harbisson about this experience and he said it was “very special.” “When someone was pointing at the passport I was actually visualising the passport,” he said. “It was not only a sense of colour, it was actually the object for me. So this is something new to me, to visualise things that are not in front of me and share someone else’s vision.”

While it’s primarily an art project for him, the ability to transmit this information across countries opens up a whole new realm of possibilities for future communication. “I can sense that I’m in New York when I’m in London, or if someone wants to send me a sunset they can do it now,” he said. “It’s communication without using sight and without using your ears. You’re not using your eyes and you’re not using ears, you’re not using touch, you’re not using smell, you’re not using taste. You’re using a new sense to communicate which opens a lot of new possibilities.”

It wouldn’t just have to be colours either; he could use the eyeborg to have a bone-conducted phone call while keeping his ears free. “We are used to maybe seeing two things at the same time but we’re not really trained to hear two things at the same time, and this is something that can be explored.”

And it sounds like Harbisson’s up for the exploration. He’s already looking toward the next upgrades to his antenna, starting with making it waterproof and removing the need for an external power source. He wants to power the device using his own body, for example using kinetic energy, brain energy, or, rather more futuristically, some sort of turbine in his bloodstream that could charge the chip as his blood flows over it.

In the future a smartphone might not even be necessary to aid communication between two eyeborgs—but for now Harbisson hasn’t managed to convince any of his friends to get their own cyborg implant and while he’s currently the first eyeborg user, he admitted he might end up being the only.

But in any case, it’s not a matter of what you have and what you don’t when it comes to the cyborg revolution. “You don’t have to have technology implanted to feel cyborg,” insisted Harbisson, comparing it to something like sexuality.

And at least if you stay off the wirelessly connected cyborg grid, you avoid any chance of someone hacking your head to, say, play loads of loud colours while you’re trying to sleep. That’s one downside Harbisson didn’t have to think about before, and he’s going to be careful who he gives his phone number to for use over the app. “Anyone now could modify my senses,” he said. “Someone could just go in my skull basically.”