After chatting for months with Google’s artificial intelligence (AI) on a school-issued Chromebook laptop, a 13-year-old student made plans to end her life, according to a lawsuit filed in California Superior Court on Nov 2, the latest in a wave of litigation over Big Tech’s effects on young people.
The lawsuit, which calls the girl Emma, said her parents first learned she had access to Google’s AI in mid-March after her school sent a message warning that she was in “imminent danger” of self-harm. Emma’s mother logged into the teenager’s school Google account and found her daughter composing a suicide note on Google Docs, the complaint said. Her parents, called John and Jane Doe in the lawsuit, immediately took Emma for an emergency psychiatric evaluation. She now goes to therapy every week.
Emma, a seventh grader in Charlotte, North Carolina, had grown convinced that she had multiple mental health disorders following interactions with AI Mode, a chatbot-like feature on Google Search, the complaint said. She also searched for “what is a suicide note” and made comments like “I am planning my own funeral” to the AI, the complaint said.
The lawsuit is likely the first to claim that Google “hastily integrated” AI features into company apps used on K-12 Chromebooks, Google-powered laptops provided to millions of children by US school districts, while failing to notify parents of potential risks. The claims add to mounting public concern that Silicon Valley has hooked young people on social media apps, video games and AI chatbots, putting children in harm’s way.
José Castañeda, a Google spokesperson, said the company had worked extensively with health experts to build features to help young people find emergency resources. The company is reviewing the claims in the lawsuit.
“For years, we’ve shown hotline information directly in search when someone may be experiencing a mental health crisis,” he said in a statement. “We’ve built those same protections into search AI features, connecting people with real-world support.”
The case underscores how public school districts have come to depend on classroom laptops, tablets and apps from companies like Google and Microsoft. In recent years, some Big Tech companies have used their own infrastructure in schools to promote their AI chatbots for teachers and students.
The new lawsuit argues that Google’s “sprawling digital ecosystem” of laptops and apps has enabled the tech giant to spread its Gemini AI chatbot across public schools – as well as turn on some students’ access to AI features by default – without parents’ awareness or permission. While school administrators can toggle company apps like YouTube and Gemini on or off for students in different grades, school administrators have complained that Google’s system has not allowed them to disable AI Mode and AI Overview, features embedded in Google Search.
Recently, Google said that it planned to automatically turn off certain AI apps and features – including its Gemini AI tool and search – in K-12 schools in early 2027. Schools that elect to turn search back on will have a choice: to enable student access to the AI features that the company has already embedded in search or to a new AI feature intended to assist students with schoolwork. Google is not offering schools an option to turn off AI in search, a company executive said, because “search and AI are deeply integrated.”
Big Tech has faced a wave of litigation this year over technology’s effects on young users. Meta, which owns Instagram and Facebook, recently agreed to pay up to US$17.1bil (RM69.41bil) to settle charges from 47 states that it used addictive features to ensnare children, following losses in similar cases. Meta has denied wrongdoing. Several parents have also sued AI companies over allegations that their children died by suicide following interactions with chatbots.
The lawsuit filed on Nov 2 on behalf of Emma and her parents makes arguments similar to some recently used against Meta, claiming that Emma was harmed by Google’s products.
Emma, who had limited access to laptops and apps at home, did not initially try to use Google’s AI on her school-issued device, according to the complaint. As Emma searched the web, the lawsuit claimed, Google “routed” her to its AI Mode.
By March, Emma was “exchanging thousands of messages” with Google’s AI Mode until late into the night and “resumed as soon as she woke up,” according to the complaint. The AI “diagnosed Emma with major mental illnesses, persuaded her to trust it over her parents and counsellors, and ultimately helped her plan her suicide and funeral,” the complaint said.
Google’s product system did not alert the school or Emma’s parents that her searches and AI chats suggested she was at risk of harming herself, according to the complaint. Her school used a digital monitoring company, which automatically scans students’ writing and searches on school devices for signs of self-harm, cyberbullying and other problems.
In an interview last month, Emma’s parents said that they learned she was in crisis only after the monitoring company flagged a suicide plan she was writing in Google Docs and warned her school. – ©2026 The New York Times Company
This article originally appeared in The New York Times.
Those suffering from problems can reach out to the Mental Health Psychosocial Support Service at 03-2935 9935 or 014-322 3392; Talian Kasih at 15999 or 019-261 5999 on WhatsApp; Jakim’s (Department of Islamic Development Malaysia) family, social and community care centre at 0111-959 8214 on WhatsApp; and Befrienders Kuala Lumpur at 03-7627 2929 or go to befrienders.org.my/centre-in-malaysia for a full list of numbers nationwide and operating hours, or email sam@befrienders.org.my.
