The parents of a 13-year-old girl sued Google on Monday in a California court, The Hindu reported on 6 November. They claim she planned a potential suicide and her funeral using Google's AI technology on a school-issued computer. The Hindu says it is the first known case accusing Google of negligence over AI on Chromebooks. Her school had blocked the Gemini app and ChatGPT. She could still use Google search, which contains AI-powered tools.
The complaint says she had no history of mental illness. On March 17, school monitoring software flagged a document titled "Funeral", and the school told her parents. The report names the document as the trigger, not her search activity. Google said it is reviewing the claims. It argues that AI mode has safeguards against self-harm and against efforts to bypass them. Andrew Liddell, one of the family's attorneys, says she was able to keep going back, and calls the safeguards insufficient. The Hindu says search's AI tools were not among those Google will turn off by default for students.
Gemini and ChatGPT are AI tools people open on their own, and the school blocked both. Search is a different tool with AI features inside it, and it stayed open. WebPulse's reading: controls set per product can miss AI built into another product. The report does not say this caused the alleged harm. Buyers can ask vendors which products embed AI, and whether each can be switched off.