Caltech scientists develop AR headset that turns the optical world into aural English

  • TECH
  • Sunday, 02 Dec 2018

Microsoft’s HoloLens virtual reality headset will make its way to some developers early next year. (Photo courtesy Microsoft/TNS)

To assist the blind in navigating unfamiliar places, scientists at Caltech in the US state of California have developed an augmented reality technology housed within a portable headset that gives objects a voice. 

This week, Caltech announced the publication of a paper ("Augmented Reality Powers a Cognitive Assistant for the Blind") outlining the work of scientists who "have combined augmented reality hardware and computer vision algorithms to develop software that enables objects to ‘talk'." 

In layman's terms, these researchers have constructed a headset powered by AR technology, called CARA, that says the name of objects in particular pitches and in particular locations to inform the wearer of what and where their surroundings are. 

CARA stands for Cognitive Augmented Reality Assistant, which is the tech that is used within Microsoft's HoloLens, a wearable headset computer that recognise objects. The tech uses "spatialised sound," or localised voices, to alert the user of where a particular object is – the closer an object is, the higher pitched "its" voice will be. 


Of course, if the scientists stopped there, the headset will be saying tons of things to the wearer simultaneously which could no doubt be pretty alarming. Instead, the team programmed CARA with three modes to make understanding a space less overwhelming. 

Spotlight mode programs CARA to voice object names only where the user's gaze is targeted at them. The second mode, scan mode, lists all objects in the environment that the headset can identify from left to right. Lastly, target mode allows the wearer to select a single object to "speak exclusively" with to use as a guide. 

Although in its early stages, the technology is promising as all visually impaired individuals who tested the headset were able to successfully navigate a route guided by CARA on the first try. With algorithms for computer vision evolving and developing at such a rapid pace, the team looks forward to the possibility of this AR headset being used at banks, hotels, and malls, basically anywhere that navigating could be a challenge for the blind, sooner rather than later. – AFP Relaxnews

Article type: metered
User Type: anonymous web
User Status:
Campaign ID: 1
Cxense type: free
User access status: 3

Next In Tech News

DBS private bank offers wealth succession planning for cryptocurrencies
So you want to delete WhatsApp? This is what you need to do first
Amazon seeks renewable power for Japan data centres - Nikkei
Op-Ed: NFTs are digital tulip fever: Would you pay thousands for a bar code?
Opinion: Telemedicine is a tool – not a replacement for your doctor’s touch
Cohen's Point72 exploring crypto sector, has no firm plans - letter
U.S. judge dismisses advertisers' antitrust claims against Google
This app purges location metadata from your photos before you share
Musk tweets, doge leaps and bitcoin retreats
Aeva announces customer deal; shares soar even after results disappoint

Stories You'll Enjoy