Real-time translation tech moves us closer to ‘language transparent’ society

  • TECH
  • Monday, 28 May 2018

A foreign-language menu hangs on the wall as an attendee demonstrates the live language translation capability of the Galaxy S9 smartphone during a Samsung Electronics Co. 'Unpacked' launch event in London, U.K., on Thursday, Feb. 22, 2018. The South Korea-based technology giant is banking on new features such as augmented reality-based emojis, camera upgrades, and stereo speakers in a form-factor similar to last year's model in order to take on Apple Inc.'s iPhone X. Photographer: Chris J. Ratcliffe/Bloomberg

Language isn't the barrier it used to be. 

Connect Google's newest headphones to a Google Pixel smartphone and the Google Assistant will translate conversations in more than 40 languages almost instantly. 

The cameras in smartphones like Samsung's Galaxy S9 can translate street signs and restaurant menus. 

Video chat software from Skype and others translates fast enough to allow two people speaking different languages to talk almost seamlessly. 

And technology developed at Carnegie Mellon University can translate lectures and speeches in real time. 

Technology is pushing us toward a “language transparent” society, said Alex Waibel, a professor and researcher at Carnegie Mellon University's Language Technologies Institute. 

“A world where we naturally maintain our individual languages but operate unimpeded by language boundaries through tech in a way as if the barriers didn't exist,” Waibel said. “And that means providing language translation delivered seamlessly in such a way that we don't notice the barrier any more.” 

Major tech companies are making major plays on translation. Facebook announced this month that its Messenger app will translate conversations in real time. Microsoft's translation app will now leverage artificial intelligence and machine learning even when it's not connected to the internet or a cellular signal. 

Amazon snagged some of CMU's top translation researchers and set up an engineering centre in Pittsburgh's South Side to develop ways to seamlessly shift Amazon products and content between languages. The team helped launch in Spanish in early 2017, shortly after Amazon opened its Pittsburgh office. 

With augmented reality, speech can be translated to text and projected onto glasses. Targeted audio speakers can direct sound so precisely that several different languages could be broadcast around the same conference room table without confusion or interference. Waibel is even working on silent speech implants, tiny devices inserted into people's cheeks and mouth that monitor the vibrations of speech so that a person can whisper in their own language and it can be translated and broadcast in any other language. 

“That's a bit of science fiction,” Waibel said. 

What isn't science fiction is what Google now offers with its Pixel Buds bluetooth headphones and the Google Assistant, the company's AI-powered answer to Alexa and Siri. Pair the Pixel Buds with a Pixel smartphone and download the Google Translate app and your phone becomes a translation station. The app will translate what you say into text and display it on the phone's screen and into speech and play it through the phone's speakers. When someone else is speaking, the phone can translate that into text on the screen and speech played into the headphones. 

The Google Translate app also comes with a camera function that will translate text on the screen. Google's camera function and a speech translation service is available on iPhones and other phones. 

Google hopes this feature will “make the world seem a little smaller”, the company wrote in response to an inquiry from the Tribune-Review. 

“It's a pretty amazing technology, and Google's working on improving it,” said Steve Van Dinter, public relations manager with Verizon, which gave the Trib a Google Pixel 2 phone and Pixel Buds to demo. “I can imagine being able to travel and having those in and how much better that experience is for someone going overseas.” 

Don't expect Google Translate or other translation technology to completely replace human translators. Alex Rudnicky, also a researcher at CMU's Language Technologies Institute, said envisions a future where technology and humans work together to translate text and speech where nuance and accuracy is important. Think Shakespeare, Rudnicky said. 

“And let's talk about poetry,” Rudnicky said. “It's evocative. It's not just for communicating meaning. It's for communicating feelings. It touches on the shared experiences of the world. 

“I can imagine, eventually, you can build systems that kind of sort of do that, but really, the easiest way to create that sort of text is to get a human in there.” 

Software and artificial intelligence also misses the gestures, facial expressions and other forms of communication that accompany and provide rich context to speech. But machine translation isn't designed to capture all that. 

“Roughly, if you consider what machine translations is used for, and will be for the foreseeable future, accurately communicating meaning is probably good enough,” Rudnicky said. 

Waibel said Google and others have playing catchup to this technology for more than a decade. 

Waibel founded Mobile Technologies in 2008 and in 2009 launched Jibbigo, a speech-to-speech translation app for the iPhone. Waibel said it was the first such app. Apple featured it in a commercial advertising the iPhone 3GS. 

“And Google said at the time it couldn't be done,” Waibel said. 

Facebook bought Mobile Technologies in 2013. Waibel and his team went to work for Facebook, full of hope that the large company could scale the technology and use it to make the world more open and connected, Facebook's mission statement at the time. 

It didn't. Team members were siphoned off to work on other projects and speech translation was left as a low priority. Waibel left. 

“I was saddened that the mission of really connecting the world wasn't at the head of things,” Waibel said. 

Waibel's technology, from early algorithms to deep-learning neural networks, has pushed computer speech and automated translation since the 1970s. He is thankful his first professor didn't laugh when Waibel told him in 1976 that he wanted to make computers convert text to speech. 

“This was really a dream of mine when I was a student,” Waibel said. 

Waibel came to CMU in 1979 to continue his research. Facebook bought a company he founded. Google has chased his innovations for the last decade. This month, Waibel travelled to Geneva, Switzerland, to talk to translators at the United Nations. Humans still interpret speeches at the UN in real time. 

Waibel's latest technology could change that. — The Tribune-Review/Tribune News Service

Article type: metered
User Type: anonymous web
User Status:
Campaign ID: 1
Cxense type: free
User access status: 3
Subscribe now to our Premium Plan for an ad-free and unlimited reading experience!

Next In Tech News

SAP to stop developing new functions for Business ByDesign software -Handelsblatt
FTX founder Sam Bankman-Fried 'willing to testify' at panel hearing - tweet
StanChart told to probe suspicious credit card charges in Hong Kong
US woman, father found dead after her toddler answers phone call from worried co-worker
The hype around esports is fading as investors and sponsors dry up
Elon Musk Twitter leak raises concern about outside data access
‘Honest grandma’, 92, repays US$3mil in debt in 10 years after refusing to declare bankruptcy wins ‘most beautiful’ citizen award in China
French President Emmanuel Macron slams TikTok algorithm over censorship, ‘addiction’
Gaming firm Razer wins lawsuit against IT vendor over data leak, awarded S$8.7mil in damages
Man tries to sell stolen bike rack on Facebook – to its owner who showed up with cops

Others Also Read