Advertisement
What we wanted to do with Helios is use next-gen technologies to radically improve accessibility for visually impaired people. It works similar to a VR set, but instead of displaying computer-generated content, the user receives real-time visual data that is captured by the Intel® RealSense™ 3D camera. Our software processes the data in order to provide the most relevant information for people with low vision. For example, if there are some obstacles nearby, our software can highlight them, and tell the user he or she is going to bump into something. It can also recognize people's faces and assist with reading too.How did you come up with the idea?
My partners Cristian Dragomir and Andrei Nistor and I took part in the Intel® RealSense™ App Challenge, which was a global competition organized by Intel to create the first applications for RealSense. It had a one million dollar prize pool, and we were actually one of the winners. We built a game called Drill Sergeant Simulator, which is when we first got to explore the full range of capabilities for the Intel® RealSense™ camera and SDK. It allowed us to get to know the technology inside out. That eventually led us towards the "Aha moment," where we knew this camera could be the secret sauce for building something to help the visually impaired.
Advertisement
We can do more on the go. Five years ago you couldn't do much on mobile. Laptop cameras were of much lower quality. We had Kinect, which is like a big hulk of equipment that had 3D vision. But the Intel® RealSense™ camera is so small, it can fit on your phone or tablet. This is the major difference from five years ago. You have the computing power and the sensors, and it's all so small and power-efficient you can actually place it in the product.What do you hope to be able to do in a few years that we just can't do yet?
What we're waiting for is the technology to shrink even more. Now we have a wearable prototype, but what we want is to have it smaller and lighter, while keeping all the performance. I think in a couple of years we'll get even better mobility from these kinds of devices.What kind of equipment is the user carrying?
The current prototype features an Intel® RealSense™ SR300 3D camera, which is mounted on a headset, and an Intel 2-in-1 hybrid laptop, which the user carries in a backpack.
From my point of view, the most challenging part was to see the world from the perspective of the people who we are trying to help: the visually impaired. It's vital for designing the product and actually giving the software features that will make sense for the user, not me or you.
Advertisement
There are some afflictions that cover most people: cataracts, glaucoma, Retinitis Pigmentosa, things like that. For the most part, they struggle to see a distant point. For them, Helios works right out of the box. The headgear has an adjustable lense, so that fixes some minor problems. We also have a few presets people can switch between in real-time, to choose whichever is best for them. For example, some people can't see at night; for them we have a nightvision mode.
Advertisement
I believe we have had several breakthrough moments while developing the software and hardware, but one of the most meaningful ones was when Mihai tried out the prototype for the first time. He just put the headset on and started using it with zero learning curve. He began describing objects in his line of sight, like tables, chairs, the closet or the door, sometimes touching them for confirmation. Then he gained confidence and went on exploring the area. Meanwhile, I was trying to hold back an ecstatic grin. I just knew we were on the right track.What if someone can't see at all? Is there anything Helios can do for them?
We actually have two versions of Helios we're working on: Helios Light is the current working prototype, which aims to enhance sight for people suffering from low vision; and Helios Touch is our concept for improving spatial awareness for the blind. The secret ingredient for both implementations is Intel® RealSense™ technology, which is used to capture visual data.The idea behind Helios Touch is to help people with complete blindness by using software that can describe their nearby environment and then communicate with them via haptic impulses. It uses vibration patterns to convey a level of information about what's around the user, somewhat like painting a picture in broad strokes. If, for instance, an obstacle is encountered, the user can feel it is there before they run into it.
Advertisement
Our long-term plan is not only to develop the best possible solution, but also make it affordable. In terms of areas we're working on, we've been focusing a lot on the human-computer interface, improving in terms of computer vision and AI, and refining the algorithms to improve organization.Though we're developing several other core projects, our greatest focus is on Helios, because we think it has the capacity to make the greatest impact. For people with low vision it's difficult, or even impossible, to do simple things like reading, writing, shopping, watching television, or recognizing faces. With Helios, all these types of activities become achievable. What we want is to empower people. Imagine people with vision impairment being able to better integrate in the workplace, or more easily access knowledge, or have improved mobility or an easy method of asking a friend for help -- this could change the lives of millions.The Intel® Developer Zone offers tools and how-to information to enable cross-platform app development through platform and technology information, code samples, and peer expertise in order to help developers innovate and succeed. Join communities for the Internet of Things, Intel® RealSense™ Technology, Modern Code, Machine Learning and Game Dev to download tools, access dev kits, share ideas with like-minded developers, and participate in hackathons, contests, roadshows, and local events.