Tencent and Alibaba’s AI models understand Chinese better than humans, new rankings show

Rivalling models from the two Chinese tech giants have achieved record-high scores on the Chinese Language Understanding Evaluation benchmark. While machines can do better than humans in some language tasks, researchers say AI still has a long way to go before it can truly understand languages. — SCMP

Artificial intelligence (AI) models from Chinese tech giants Tencent Holdings and Alibaba Group Holding understand the Chinese language better than humans, according to a benchmark test measuring natural language processing (NLP).

The two rival models have achieved record-high scores on the Chinese Language Understanding Evaluation (CLUE) benchmark, which is a set of tasks designed to evaluate how well a machine can understand and respond to Chinese text similar to how humans do.

This marks the first time that AI models have scored better than humans on CLUE since the benchmark was established by dozens of researchers three years ago.

Tencent’s “Hunyuan AI model” came first with a score of 86.918, followed by Alibaba’s AliceMind with a score of 86.685, according to Monday’s rankings on CLUE’s website. Both ranked higher than humans, which have been given a score of 86.678.

AI models from Chinese smartphone maker Oppo and food delivery giant Meituan ranked fourth and fifth, respectively.

“Although the rankings change a lot, the top scorers had never surpassed humans until now,” said Alibaba Cloud – the cloud computing arm of the ecommerce giant – in an article published on Friday. It added that the latest results mean that “the Chinese language understanding of the AI model has reached a new level”.

Alibaba owns the South China Morning Post.

Alibaba’s Tmall Genie smart speaker. Photo: Handout

China’s Big Tech firms have been working to improve their NLP technology, which is used to support voice-enabled virtual assistants, such as Alibaba’s AliGenie and Tencent’s Xiaowei, as well as other functions like machine translation and spam detection.

Earlier this year, Chinese internet search giant Baidu said that its bot Du Xiaoxiao wrote an article that scored higher than most students on China’s notoriously difficult national college entrance examination, known locally as gaokao, although it mistakenly used an internet slang term.

Still, some researchers say most AI models still have a long way to go before they can truly understand the intricacies of languages.

Last year, scientists at Auburn University in the US state of Alabama and the research unit of American software company Adobe found that many AIs that performed better than humans on certain comprehension tasks were unable to tell when words in a sentence were randomly shuffled. – South China Morning Post

Article type: free
User access status:
Subscribe now to our Premium Plan for an ad-free and unlimited reading experience!

Next In Tech News

Opinion: Teens carry a threat to mental health in their pockets
Like Terminator’s T-1000, this robot liquefies before returning to its original form
This app offers travellers on the London Underground less-polluted routes
Find out if your personal data has fallen into the wrong hands online
FTX founder Bankman-Fried objects to tighter bail, says prosecutors 'sandbagged' him
As they enter a 4th generation, are foldable phones finally mature?
This free tool lets you extract text from images
Google Stadia is dead, but its controllers live on
Twitter says users will be able to appeal account suspension
New smart-home standard for Android and Google devices has arrived

Others Also Read