AI-generated receipts. Software companies that audit expense reports are adding a new arsenal of capabilities to try to detect receipts that have been created using AI chatbots. — The New York Times
It wasn’t long after ChatGPT began generating realistic images that Anant Kale started seeing posts on social media that explained how it could be used to generate a pretty convincing fake receipt.
That was, he recognised, his problem.
As the CEO of AppZen, a software used by finance teams to manage expenses, he had overseen the creation of fraud-detection tools that flagged artificial-intelligence-generated receipts. But this was different.
“We were like, oh, shoot, this is too easy,” he told The New York Times.
AppZen immediately started developing a tool to detect fake receipts generated by chatbots. It’s not the only one: The expense management app Expensify added ways to detect AI-generated receipts in April, and SAP Concur’s automated expense-auditing tool, Verify, expanded a similar capability to all users this month.
This summer, when announcing new efforts to flag AI-generated receipts, Nicolas Ritz, who works on product development at the corporate travel software company Navan, summed up the dilemma: “AI-generated receipts will only get better from here,” he wrote. “To combat fraudulent AI, we need to use AI.”
Expense fraud can be a slippery slope. Kale said it’s common for employees to generate their first fake receipt to account for a legitimate expense. Maybe they lost the receipt. But when they don’t get caught, they do it again. Occasionally, the fraud is egregious. AppZen once detected a batch of AI-generated receipts submitted by a company employee for hotels and airfare in Bangkok – a city that, upon further investigation, the employee had not visited.
The Association of Certified Fraud Examiners, which certifies about 5,000 new examiners each year, regularly asks members to submit the largest case of occupational fraud they’ve investigated in the last 18 months. In the most recent survey, about 13% of the cases involved employees who submitted inflated or invented expenses, which can lead to criminal charges. The median loss was $50,000.
Fake receipts make it easier. About 30% of fraudulent receipts that AppZen catches are now generated by AI chatbots, rather than through an image editor or a template service, the company said, and the number of fraudulent receipts it catches overall has increased about 30% since May 2024. Expensify said it detects hundreds of AI-generated receipts each month, out of the millions of receipts it processes. SAP Concur flagged about 1% of receipts audited by Verify as potentially generated by AI.
“I have definitely heard from members and other anti-fraud experts that AI is directly resulting in not only an increase in this type of fraud by volume but also making this type of fraud more difficult to detect,” said Mason Wilder, the research director at the fraud examiners association.
Convincing fakes have sparked a new tech escalation. Not entirely dissimilar from when, say, home printers created a need for new expense-fraud-detection methods, software companies have built an arsenal of methods for catching a new tier of fake receipts generated by AI.
Chatbots leave a fingerprint in the metadata of the images they generate, but if an employee takes a photo or screenshot of the image, that signal disappears.
An algorithm can compare AI-created receipts with real receipts from the same vendor. It might pick up on slight differences in font or spacing, for example, that a human eye wouldn’t.
Like many expense-auditing software tools, AppZen has relied on identifying suspicious patterns – like spending that is unusual for the time of day or employee’s role – to flag receipts that warrant a closer look. Those suspicious receipts are submitted to its newer second layer of auditing, which looks for patterns that signal that a chatbot may have produced them. While generating restaurant receipts, for example, did the employee always ask the chatbot to use the same server name or dish order?
It’s not a single technique that can detect such receipts, Kale said: “It has to be layers and layers. It’s a cat-and-mouse game.” – ©2025 The New York Times Company
This article originally appeared in The New York Times.
