Over the past year or two, companies have started using so-called artificial intelligence (AI) agents as bona fide “employees,” even including them in their organisational charts.
Emma Wiles, a Boston University professor who studies how AI affects workers, stumbled onto this phenomenon in October, at a conference where two human resources executives said that treating AI agents like real employees was a way to increase productivity and to put their companies on the cutting edge.
But when Wiles and three collaborators from Boston Consulting Group investigated further, they discovered a pitfall. In an experiment involving dozens of companies with AI employees, the researchers found that managers tended to vet documents less carefully when told an AI employee had produced them. The managers missed errors that other managers caught when told they were vetting the work of a human.
Wiles speculated that managers didn’t think sussing out mistakes made by AI employees was their responsibility. If something went wrong, they could dismiss it as the fault of the tech team or of the executives who wanted AI employees in the first place. “But it’s not your problem,” she said, channelling the managers’ mindset about their own roles.
In the years since AI burst onto the scene, many companies have become aware of flaws produced by the technology and, at times, taken steps to offset them. They know that AI models can be biased against certain groups of people, including nonwhites. They know that chatbots can provide confident but incorrect answers to queries. They know that the bots sometimes spill the beans on information that should remain private.
But as companies race to bring AI into their day-to-day operations, researchers are discovering more subtle defects. In principle, these flaws could be corrected, too. For example, companies could hold managers directly responsible for the mistakes of AI subordinates.
But in practice, most corporate users appear to be blissfully unaware of these issues, raising the possibility that AI’s promise of increased productivity and vast cost savings could be undermined.
Even researchers who study AI may be aware of only a fraction of the problems that the technology introduces. “There are a whole host of unknown unknowns,” Wiles said.
One well-documented but underappreciated flaw of artificial intelligence models is that they tend to favour work produced by AI. A 2025 paper in the Proceedings of the National Academy of Sciences found that several large language models had a low opinion of text written by humans, creating a “potentially consequential form of implicit ‘antihuman’ bias.”
But many companies seemed unaware of this problem, or at least unable to imagine how it might wreak havoc on their operations. When a team of scholars spelt it out in a subsequent paper, finding that the AI models that companies use to evaluate resumes tend to favour those written with the help of AI over those written entirely by humans, it caught the attention of some corporate recruiters.
Jane Yi Jiang, an operations professor at Ohio State University who is an author of that subsequent paper, said that she and her coauthors were happy to help when recruiting firms inquired about “how to improve their processes.”
But they noted that this was almost certainly not the only problem companies were inadvertently introducing in their rush to adopt AI. “People are moving so fast to use LLMs without thinking too much about the implications, biases,” she said, referring to large language models like AI chatbots.
For example, some companies now use AI to help answer questions like how much to charge for a product or where to open a new location. Relying on the technology for such purposes, however, can quickly go off the rails.
When left to their own devices, humans often cooperate and seek win-win outcomes. But when AI models assess a situation, they tend to adopt the more coldly calculating, “rational” mindset that arises from basic game theory. They might, say, lead a company to aggressively undercut a competitor, even though it risks a damaging price war.
“Most of the LLMs we test think that human beings are more rational than they actually are,” said Jiannan Xu, a doctoral candidate at the University of Maryland and collaborator of Jiang’s. “But the most rational response leads to a bad situation for all” in many cases.
Wiles, the Boston University professor who examined the way humans manage AI employees, said the shortcomings weren’t necessarily intrinsic to the technology, but arose when humans adopted it with little attention to what could go wrong.
She and her colleagues surveyed more than 1,000 corporate managers and found that about one-third said their organisations referred to AI as a “teammate or employee,” and that nearly one-quarter said their employer included AI agents on its organisational charts. “We call it Scout,” one manager told the researchers in an interview, referring to an AI agent. “It’s technically an equivalent peer on your team.”
Wiles and her colleagues gave all the managers they surveyed a set of five documents that contained errors, and gave them 20 minutes to review as many as possible. In some cases, they told the managers that an AI employee had done the work; in some cases, they said that an AI tool had done the work; and in some cases, they said that a human had done the work.
In general, the stated source of the documents didn’t make much of a difference in how closely managers vetted them.
But managers at companies that included AI agents on their organisational charts caught substantially fewer mistakes when told they were reviewing the work of an AI employee.
People who manage humans tend to assume that “if someone on my team makes a mistake, that’s on me,” Wiles explained, which is why they closely check the work of these subordinates. Managers also seem to assume that they’re on the hook for work produced by an inanimate AI tool. But managers at companies with AI employees don’t seem to feel the same responsibility for the work of those employees.
Her takeaway: Over the past few centuries, scholars and business leaders have developed a reliable set of practices for managing humans. But the psychology of managing anthropomorphised AI is vastly different, and “we’re going out there blind.” – ©2026 The New York Times Company
This article originally appeared in The New York Times.
