You need to quickly learn about a new topic. Or you need an answer to an obscure question. Should you use a normal search engine or should you have an AI tool such as ChatGPT or Claude do the searching for you?
A new study from Stanford University offers some answers to those questions. In the study, researchers asked ChatGPT, Gemini, and Claude some multiple-choice questions about recent events. They were specifically testing AI’s effectiveness as a news source, because more and more people are using it that way. But the strengths and weaknesses they uncovered could apply to any type of query.
In an article for Wired, tech journalist David Nield used those results, along with some other recent research, to create a brief guide to when to search using AI and when not to. It’s well worth checking out. Combining his advice, the Stanford research, and my own long experience with online research, here’s my take on which tool you should use and when.
Use AI if…
You want a general sense of something. I knew a lot of people considered sourdough bread to be healthier than traditional yeast bread, but I wasn’t sure why they thought so. AI can answer a question like that really well. (Due to sourdough’s long fermentation process, it’s easier to digest, causes less of a glucose spike, and allows for better absorption of iron and other minerals.) I also find AI to be useful for general how-to information. It’s given me helpful gardening and cooking advice, for example.
You want information that’s widely known. Nield uses the height of the Empire State Building as an example. Because that is accurately reported in many different places, the odds are high that AI will give you the correct answer.
You need something complex explained in plain English. A friend of mine was diagnosed with prostate cancer. There was a ton of information about different test results and antigen levels. It was all a little overwhelming. Then he entered some of this info into Google Gemini and it gave him a straightforward description of what the medical info meant. He had never used AI before, but he was an instant convert.
You have a question with multiple parameters. Nield’s example is a family of five looking for dinner in San Francisco for less than US$200 (around RM817). But it can also help with lead generation. I know speakers who ask AI to list the associations in their area that cover their topic and are having events between six months and a year in the future, for example. Search engines can’t handle several elements at once in this way.
Use traditional search if…
Your question is unusual or obscure. Large language models like ChatGPT or Claude are probability engines. When you ask for information, they aggregate information from a wide range of sources to find the most probable correct answer. There’s always a danger of an AI delivering a wrong or fictitious answer it got from an untrustworthy source. That danger increases when there are only a few sources to choose from, and AI is liable to trust the wrong one.
You want to be sure of the sources. You can specify that you want AI to use reliable, authoritative sources for its information, but it isn’t always good at judging what is and isn’t a reliable source. And, Nield reports, some large language models choose answers that seem most relevant to the questions asked, even if that means preferring dubious sources over reliable ones.
A further problem is that some reliable sources, such as many publications, block AI bots from scraping their material for copyright infringement reasons. A traditional search engine will still lead you to those sources.
You are searching material that isn’t in English. As you likely know, there’s much more information online in English than in any other language. Although large language models are multilingual, they can only draw on information the internet supplies. That can lead to US-centric bias, and sometimes incomplete or inaccurate information. The Stanford research found that questions asked in Hindi about news in Hindi-speaking regions of India were much more likely than others to generate wrong answers. In part, that’s because when faced with local questions, AI often defaulted to the English-language version of Wikipedia as a source.
When it absolutely has to be right. Given AI’s inherent limitations, you should always double-check the info it gives you. AI information should include links to sources (you can ask for them if it doesn’t). You should actually click on those links and find the information for yourself. I’ve often had AI cite a web page as a source, only to find that the page didn’t actually say what the AI reported.
AI is a powerful research tool, one I’m learning to use more and more. But it has serious limitations, too. Knowing when to use AI and when not to can help you put that power to work for you. – Inc./Tribune News Service
