AI tools have widely been adopted by programmers in efforts to speed up production, however new research is casting doubt on the assumption that it generally helps developers work faster. — Photo: Sebastian Gollnow/dpa
LOS ANGELES: Open-source developers who lean on artificial intelligence tools are less productive and efficient than those who do not use the much-hyped technology, according to researchers investigating AI assistance in software development.
Using AI means the work takes 19% longer to finish than without, said a team at METR (Model Evaluation and Threat Research) after a randomized control trial involving 16 developers who undertook a series of typical tasks their job entails, such as bug fixes.
AI tools have widely been adopted by programmers in efforts to speed up production, and in May ChatGPT maker OpenAI released a dedicated software engineering agent called Codex for software developers.
However: "AI makes them slower," the Californian non-profit METR said, deeming their outcome surprising - as did the developers, who said in advance that they expected using AI to make them around 25% faster at getting through their tasks.
Not only was the "significant slowdown" contrary to developer beliefs and experts' forecasts, the "gap between perception and reality" was "striking," according to the researchers.
Perhaps the most striking finding of all was that despite AI throwing sand in their gears as they worked, the developers nonetheless believed it "had sped them up by 20%," according to METR’s team.
"They were mistaken about AI’s impact on their productivity," the METR team said of the participants.
But the researchers by no means wanted to rule out AI as a helpful tool for coders and said it could still be useful for less experienced developers or those "working in an unfamiliar codebase."
The findings echo research published by Stanford University earlier this year showing AI as largely hindering workers of experience, while it boosts those with less skill. The effects of AI on productivity "vary significantly," the researchers said. – dpa
