Many professionals are tempted to use AI chatbots like ChatGPT to optimize their performance and increase their efficiency at work. However, US research published in the Financial Analysts Journal warns financial specialists against relying too heavily on these artificial intelligence tools.
A research team led by DJ Fairhurst of the Carson College of Business at Washington State University studied over 10,000 answers generated by AI models such as ChatGPT, BARD and Llama in response to financial exam questions. These tools not only had to answer questions taken from licensing exams, but also had to justify the reasoning behind them. The results were then compared to answers given by human industry professionals. "Passing certification exams is not enough. We really need to dig deeper to get to what these [AI] models can really do," explains study author, DJ Fairhurst, quoted in a news release.
ChatGPT 4.0 – the paid version of OpenAI's AI chatbot – stood out with an accuracy rate 18 to 28 percentage points higher than the other AI models tested. But ChatGPT 3.5 proved to be more accurate than its successor, after being trained with examples of correct responses and explanations.
However effective these tools may be, they are still far from rivaling human expertise, especially when it comes to complex issues. While they excel in routine tasks, such as analyzing financial transactions, they show their limits when it comes to more technical subjects, such as assessing insurance coverage or handling a client's tax status. "For broad concepts where there have been good explanations on the internet for a long time, ChatGPT can do a very good job at synthesizing those concepts. If it’s a specific, idiosyncratic issue, it’s really going to struggle," says DJ Fairhurst.
In their study, the researchers conclude that these artificial intelligence tools should be seen as complementary aids rather than substitutes for the expertise of human professionals. They could, however, transform certain finance jobs, particularly those of junior analysts often assigned to repetitive tasks.
Although AI models do not yet represent a threat to skilled jobs, they are already redefining the contours of several professions. The question remains whether they will one day succeed in overcoming their current limitations, or whether they will remain confined to roles supporting human workers. – AFP Relaxnews
