Health IT, Startups

Exploring the potential of VR and AI-enabled tools for the visually impaired

One area of interest for augmented and virtual reality applications is making the world — and virtual world — more accessible to blind and visually impaired people.

data, patient, medical records, health data, healthcare data

One area of interest for augmented and virtual reality applications is making the world — and virtual world — more accessible to blind and visually impaired people. Some companies have already developed applications in this area such as Samsung, OxSight, and Aira. A group of Microsoft researchers sought to add a new twist to the canes used by visually impaired people to navigate by adding augmented reality and haptic feedback.

In a paper discussing their approach, the researchers described the current landscape as dominated by companies and researchers enlisting an audio-based experience to help visually impaired people navigate. With their Canetroller device, the researchers sought to move the needle beyond these approaches and consequently expand and deepen visually impaired people’s interaction with their environment:

Canetroller provides three types of feedback: (1) physical resistance generated by a wearable programmable brake mechanism that physically impedes the controller when the virtual cane comes in contact with a virtual object; (2) vibrotactile feedback that simulates the vibrations when a cane hits an object or touches and drags across various surfaces; and (3) spatial 3D auditory feedback simulating the sound of real-world cane interactions. We designed indoor and outdoor VR scenes to evaluate the effectiveness of our controller.

Although the paper sought to highlight the potential of its own take on the white cane, the researchers acknowledged that the current generation of smart white sticks are not widely sought because they don’t yet have the confidence of visually impaired people that they work reliably.

To evaluate its device, the researchers recruited nine people, six of whom participated in formative and system evaluation studies. The studies evaluated the Canetroller in indoor and outdoor virtual settings unfamiliar to the participants. The indoor study found participants could locate virtual objects and could recall where they were. The haptic feedback component acted like a “braking” mechanism for the cane and helped alert users of physical barriers and virtual barriers. Although the indoor virtual world gave participants a strong sense of location, the outdoor virtual world proved more disorienting, particularly because the direction of the traffic sound was tough for some participants to identify the direction the sound came.

The researchers concluded that the device has the potential to be used for entertainment, orientation and mobility training, and environment preparation.

An interesting wrinkle in the development of VR and AR-enabled technologies to support the visually impaired is that some of these technologies have matured over the years. OrCam is a good example. This week, Reuters reported that OrCam raised $30.4 million by selling a 3 percent stake to investors, including Israel’s Clal Insurance, bringing it to more than $130 million raised to date. The company rolled out an update of its smartglasses with a camera that can read material such as texts, supermarket barcodes and recognize faces while speaking relevant information into the user’s ear, according to Reuters.

Aira has also advanced the scope of its own product by identifying environments where it is critical for people to find their way around rapidly but is inevitably unfamiliar and stressful — the airport. A number of airports have adopted Aira’s navigation platform for the visually impaired, which provides Site Access Technology through an app and a set of smartglasses.

Photo: nevarpp, Getty Images