AI can recommend; humans must decide

Image: Magnific

A PATIENT lies unconscious in an intensive care unit. Days have passed without improvement.

An artificial intelligence system, having analysed thousands of similar cases, indicates that the likelihood of recovery is extremely low.

It recommends withdrawing life support.

The medical team reviews the analysis. The conclusion appears statistically sound.

Yet, standing at the bedside, the decision does not feel complete.

What remains is not a question of accuracy but of responsibility.

In moments like this, the limits of AI become more visible.

It is capable of processing vast amounts of information and identifying patterns that would be difficult for any individual to discern.

It can suggest what appears to be the most optimal course of action based on available data.

In many fields, from medicine to finance to public policy, such capabilities have brought tremendous benefits.

However, decisions are not defined by correctness alone.

They carry consequences that extend beyond the immediate outcome, affecting lives, relationships, and at times, the moral integrity of those who must act upon them.

For the patient in the intensive care unit, the question is not only whether the recommendation is statistically justified but also who is prepared to accept what follows from that decision.

This distinction is important.

AI can recommend, analyse and even predict with remarkable precision, but it does not bear the weight of the decisions it informs.

It does not stand beside the patient’s family. It does not experience doubt, nor does it live with the consequences of what is decided. That responsibility remains with human beings.

Not long ago, the relationship between judgment and responsibility was more clearly aligned.

A physician made a diagnosis and stood by it. A judge delivered a verdict and accepted accountability for the decision. A manager made a hiring choice and carried the implications.

Their decisions were shaped by knowledge, experience and an awareness that they would be answerable for the outcomes.

Today, this alignment is becoming less distinct.

As algorithms become more influential, there is a growing tendency to treat their recommendations as objective or neutral.

Because they are derived from large datasets and complex models, they can appear authoritative.

Yet, every system reflects human choices – from the data it is trained on to the objectives it is designed to optimise. These choices shape the recommendations that emerge.

More importantly, the system itself remains detached from what follows. This raises a subtle concern. When decisions are increasingly guided by systems that do not share in their consequences, there is a risk that human beings may begin to distance themselves from the weight of those decisions.

The process becomes more efficient, but the sense of ownership may be less immediate.

Judgment develops through experience, reflection and an awareness of the human implications of a decision. It involves not only asking what can be done, but also considering what ought to be done and who will be affected by it. It carries with it a willingness to accept responsibility even when the outcome is uncertain.

As the mathematician Norbert Wiener observed many years ago, we must be certain that the purposes we embed in our machines are truly the purposes we intend to serve. His warning remains relevant not only at the level of system design but also in how we relate to the systems we use.

As AI continues to evolve, the challenge before us is not simply to build more capable systems but to remain attentive to what those systems cannot do.

They can guide, support and recommend. But they cannot carry what follows.

NG KWAN HOONG

Emeritus Professor of Biomedical Imaging

Faculty of Medicine

Universiti Malaya