One of us got a call last spring from a longtime friend.
The story was familiar: two doctors, an MRI (magnetic resonance imaging), an online AI (artificial intelligence) tool, a stack of articles – and one anxious question.
“Everything tells me something different. The AI says I might need surgery. What should I do?”
We believe there’s one key response to anyone in this all-too-common conundrum: “What matters most to you?”
There was a long pause.
What you want
That pause is one of the most important moments in modern healthcare – and it is exactly the question AI is unable to address.
In our careers as physicians and researchers, we have found, clearly and repeatedly, that for many common conditions, the medical evidence does not point to a single “right” answer.
The biology is often close.
What determines the success of an outcome is whether the choice fits the person making it.
Some patients with back pain want the fastest possible return to physically-demanding work, even if it means surgery.
Others want to avoid an operation at almost any cost, even if recovery takes longer.
The scan may look the same.
The lives behind the scan are not.
That insight is becoming critically important as AI moves deeper into everyday health decisions.
In our research on AI and clinical decision-making, we’ve studied what happens when systems are trained to optimise medical outcomes, but are blind to human values.
In plain English, today’s AI is very good at telling you what usually works for people like you with similar demographics and medical histories.
It is far less capable of understanding what you are trying to protect, avoid or prioritise.
Small differences
This matters because some of the most common and most expensive medical decisions are not purely biological.
Should someone with low-risk prostate cancer choose surgery, radiation or careful monitoring?
Should a person with atrial fibrillation undergo a procedure or manage the condition with medication?
Should a patient with chronic knee or back pain operate now or try months of physical therapy to see whether surgery can be avoided?
In these situations, the medical differences between options are often small or uncertain.
What makes the biggest difference is whether the treatment aligns with the patient’s goals: tolerance for risk, willingness to undergo recovery, ability to adhere to long-term therapy or simply what kind of life they want to live.
AI systems can calculate probabilities; they cannot determine what those probabilities mean to a particular person.
Knowing what matters
In some respects, AI may know more medicine than any individual physician.
It can synthesise millions of scientific papers, clinical studies and patient records in seconds.
Yet, it knows remarkably little about the person sitting across from it.
AI does not know a patient’s goals, fears, obligations, tolerance for risk or personal definition of a good outcome.
And because it knows little about either the patient or the physician, it knows even less about the conversation between them – the place where facts, values and trust come together to produce the right decision for a particular person.
A second patient story brought this home.
A retired teacher was referred after an AI-based symptom checker flagged a heart rhythm abnormality and “favoured” an invasive procedure.
The patient arrived frightened, convinced there was one correct path.
When we talked, it became clear that what mattered most was avoiding a long recovery and staying healthy enough to travel to see grandchildren.
Medication and monitoring – less dramatic, but well-supported by evidence – fit those goals better.
The AI wasn’t wrong; it just didn’t know what mattered.
Take time to reflect
This blind spot is not trivial.
Roughly a quarter of US healthcare spending flows through decisions in which patient preferences meaningfully affect outcomes.
When those preferences are ignored – by people or by algorithms – care becomes misaligned.
That can mean unnecessary procedures, poor adherence, regret and rising costs without better health.
So what should consumers do when an app, portal or “smart” tool recommends a course of action?
Start with three questions.
First: “Best for whom?”
If a tool says one option is best, ask whether it means best on average or best for someone with your priorities.
Second: “What does this system not know about me?”
AI can see lab values and imaging results.
It cannot see your job, your family responsibilities, your fears or what you are trying to get back to.
Third: “What happens if I wait or choose differently?”
Many important medical decisions are not emergencies.
When options are close, taking time to reflect is often part of good care.
AI is becoming a powerful partner in medicine.
It can help explain options, surface evidence and reduce confusion.
But it should inform human decisions, not replace them.
AI may know more medicine than any physician.
It knows far less about any patient.
And it knows least about the conversation between them.
The most important variable in your healthcare is not in any algorithm.
It is you. – By Prof Dr James N. Weinstein and Asst Prof Dr Ogan Gurel/Los Angeles Times/Tribune News Service
Dr James N. Weinstein is a surgeon, Northwestern University clinical professor and Health Futures global head at Microsoft, which develops AI systems. Dr Ogan Gurel is a physician and University of Texas at Arlington assistant professor, where he researches AI, causal inference and patient decision-making.
