The spread of artificial intelligence (AI) is almost as rampant as the flu in an urgent-care waiting room, but the adoption of the technology in the health care industry may be the reason why you have to wait a bit longer to be seen.
A new study from Dartmouth College found that AI mistakes can cost doctors vital time when filling out charts. Errors and irrelevant details are forcing physicians to spend time correcting AI-drafted responses, time that could be spent treating or speaking with patients.
The study was presented at the 2026 Annual Meeting of the Association of Computational Linguistics and published in the conference proceedings. This was the first large-scale study of an online patient portal that utilises AI to draft responses to patients.
The researchers developed a tool that compares AI-generated responses to a dataset of real responses crafted with health care professionals from Dartmouth Health. They analysed 146,000 conversations between 10,105 patients and their primary care physicians at the large, rural health care system. Additionally, they used the tool to evaluate physician responses by Claude, Gemini, ChatGPT, Llama, Aloe, and Qwen.
“We find that AI can sound like a doctor but not think like one,” Sarah Preum, PhD, corresponding author of the study, said in a release.
AI and misalignment
The findings suggest that AI-generated responses are often misaligned with what the physicians would actually write. That includes responses that are too long, include irrelevant or inaccurate medical details, or lack follow-up questions.
In one instance, the portal’s AI suggestion to a 32-year-old woman taking an acid reflux drug who was concerned about constant nausea was that her diet should be adjusted. The physician overrode that suggestion and asked if there was any chance of her being pregnant.
These slight changes can produce a bottleneck effect, adding up hundreds of little edits to thousands of messages, slowing down the process.
Of all the gaps the researchers identified, the failure to ask clarifying follow-up questions stood out. That’s an issue.
Preum told Inc that the follow-up question is often what steers care in the right direction.
This is especially true for symptom-reporting messages, where asking the wrong question, or none at all, can send the patient down the wrong diagnostic or treatment path. The stakes are even higher for vulnerable populations, Preum said, including older adults, patients managing multiple chronic conditions, people on immunosuppressive or cancer treatment, and pregnant patients.
“A model can always generate an answer without asking anything first – that’s not the failure mode,” Preum told Inc. “The failure mode is that a real clinician, given the same message, would have asked a clarifying question before responding. When a model skips that step, it’s not being efficient, it’s guessing.”
In the Dartmouth dataset, clinicians asked an average of two follow-up questions per symptom-reporting conversation before responding.
Individualised message generation
However, there are some potential benefits to the new technology in health care
Researchers found that by adapting AI to individual physicians’ communication styles, accuracy can improve by 33% and reduce editing by up to 26%.
The study found that AI responses can be helpful when customised to physicians’ needs. Researchers created a technique called Tadpole – Thematic Agentic Direct Preference Optimisation for Learning Enhancement – that trains AI platforms using a hybrid model constructed from the physician and AI-generated responses.
When they plugged Tadpole into the six commercial LLMs, they found that drafted responses better matched the physician’s standard for precision and information quality, saving busy clinicians one to two hours of work each day.
Yet, even as health care systems start to roll out AI draft replies in patient portals, it may need more rigorous testing.
Deploying these tools at scale before they’ve been evaluated for safety, bias, and other responsible-AI practices is a real danger, especially for vulnerable patient populations. It also presents a potential danger for providers and health systems; a wrong or misleading AI-generated response leaves a great liability risk. – Inc./Tribune News Service
