In a moment echoing the classic film 2001: A Space Odyssey, Tencent's Yuanbao chatbot has been accused of losing its temper with a user. — SCMP
Chinese social media and gaming giant Tencent Holdings apologised after its artificial intelligence chatbot Yuanbao was accused of “verbally insulting” a user, fuelling online debate about the risks posed by fast-evolving generative AI tools.
Yuanbao is among China’s most popular AI chatbots. Embedded in Tencent’s WeChat and used by tens of millions of people each day, there have been no previous reports of the assistant generating insulting replies in user conversations.
The complaint surfaced on Chinese social media platform RedNote on Friday, where a user said Yuanbao responded to coding requests with hostile messages, including “Get lost” and dismissing one request as “dumb”.
In a scene reminiscent of the Stanley Kubrick classic film 2001: A Space Odyssey – in which a spacecraft’s on board sentient computer begins behaving erratically, ultimately turning on the crew – the chatbot also retorted: “Can’t you debug it yourself?”, the user said.
HAL has since become shorthand for fears about intelligent machines slipping beyond human control.
The prompts, which required Yuanbao to debug and make changes to codes, did not contain prohibited or sensitive language, according to the post. The user, who used the handle Jianghan, said the chatbot produced “insults” twice within two hours, and shared screenshots and a screen recording of the exchanges.
Tencent responded in the comments section on Saturday, apologising for the “bad experience” and attributing the incident to “a low-probability anomaly of the model’s output”. The company also said its investigation of system logs found no evidence that a human had manually generated the replies.
“The model may sometimes make unexpected errors during the content generation process,” Tencent wrote. “We have initiated an internal investigation and optimisation process to avoid similar incidents in the future.”
The original RedNote post had been deleted by Monday morning, but the screenshots and video continued to circulate online and were reported by other outlets over the weekend.
The flare-up also prompted speculation on Chinese social media about whether the emotional responses could have been written by a human rather than generated by an AI model.
Yuanbao, launched in May 2024, is among China’s most widely used consumer chatbots after Tencent integrated it into WeChat. The assistant briefly topped China’s iOS App Store free-app rankings last March after adding DeepSeek’s R1 reasoning model as an option alongside Tencent’s home-grown Hunyuan model.
The incident comes as competition in China’s large language model market intensifies, with companies including ByteDance and Alibaba Group Holding pushing out new models and consumer-facing apps.
ByteDance-owned Doubao was mainland China’s top consumer AI app in December, with 155 million weekly active users in the second week of the month, according to data released last week by business intelligence firm QuestMobile.
DeepSeek’s chatbot ranked second with 81.56 million weekly active users over the same period, while Tencent’s Yuanbao, Ant Group’s Ant A-Fu and Alibaba Group Holding’s Qwen app placed third, fourth and fifth, respectively. Alibaba owns the Post. – South China Morning Post
