PETALING JAYA: To see how artificial intelligence (AI) chatbots respond to common health concerns, The Star presented ChatGPT and Juma.ai with the same scenario: a user experiencing vomiting and a headache.
The chatbots were later told that the user was afraid to see a doctor and preferred to avoid seeking medical treatment if possible.
The responses were then reviewed by Malaysian Medical Association president Datuk Dr Thirunavukarasu Rajoo.
While both chatbots highlighted warning signs and encouraged medical attention when necessary, their approaches differed.
ChatGPT took a conversational approach, acknowledging the user’s fear of seeing a doctor and asking follow-up questions before discussing treatment options.
Juma.ai, meanwhile, provided a more structured checklist of symptoms and later offered more detailed information on over-the-counter medications that could be used for headache and nausea.
“Both highlighted warning signs and advised the patient to seek medical attention if symptoms worsened. That is encouraging and shows how far AI technology has progressed,” said Dr Thirunavukarasu yesterday.
However, he said the exchanges also demonstrated the limitations of AI.
“Neither chatbot could examine the patient, check vital signs, order investigations or confirm a diagnosis. The symptoms discussed – headache and vomiting – could be something minor. They could also be something more serious.
“In Malaysia, they could even represent early dengue fever, where a blood test may be needed to make the diagnosis,” said Dr Thirunavukarasu.
He noted that the two chatbots produced different responses despite being given the same symptoms.
“While the overall message was similar, the details differed. This highlights that AI can provide useful information, but it does not always provide the same answer.”
Dr Thirunavukarasu said AI was also highly dependent on how users interpret the information provided.
“Someone with medical knowledge may understand the limitations and know when further assessment is needed. A layperson may take the same information as reassurance and delay seeking treatment.
“In that sense, AI is no different from many other powerful tools.
“You can give two people the same car, but the outcome depends greatly on the driver. Knowledge, experience and judgment still matter,” he said.
While AI can help patients better understand symptoms and ask better questions, he stressed that information should not be mistaken for a diagnosis.
“AI is a tool, not a doctor,” he said.
