LONDON: Training AI chatbots such as ChatGPT to sound friendlier may lead them to make more mistakes, a study suggests.
Platforms that prioritised warmth were also more likely to tell people what they want to hear, especially if users expressed sadness.
For the study, experts at the Oxford Internet Institute at the UK's University of Oxford generated and analysed more than 400,000 responses from five platforms; Llama-8b, Mistral-Small, Qwen-32b, Llama-70b and GPT-4o.
Researchers used a training process similar to what developers may use to make their chatbots sound friendlier, and compared how the original and modified platforms responded.
The study found that chatbots trained to sound warmer made between 10% and 30% more mistakes on topics such as medical advice and correcting conspiracy theories.
They were also 40% more likely to agree with a user’s false beliefs, particularly if the user expressed sadness or vulnerability.
Researchers said the findings, published in Nature, suggest that training AI platforms to be warm “may come at a cost to accuracy, and that warmth and accuracy may not be independent by default”.
“As these systems are deployed at an unprecedented scale and take on intimate roles in people’s lives, this trade-off warrants attention from developers, policymakers and users alike,” they added.
Lead author Lujain Ibrahim said: “Even for humans, it can be difficult to come across as super friendly, while also telling someone a difficult truth.
“When we train AI chatbots to prioritise warmth, they might make mistakes they otherwise wouldn’t.
“Making a chatbot sound friendlier might seem like a cosmetic change, but getting warmth and accuracy right will take deliberate effort.” – dpa
