Your New Therapist: Chatty, Leaky, and Hardly Human

If you or someone you know may be experiencing a mental health crisis, contact the 988 Suicide & Crisis Lifeline by dialing or texting “988.” Vince Lahey of Carefree, Arizona, embraces chatbots. From Big Tech products to “shady” ones, they offer “someone that I could share more secrets with than my therapist.” He especially likes the apps for feedback and support, even though sometimes they berate him or lead him to fight with his ex-wife. “I feel more inclined to share more,” Lahey said. “I don’t care about their perception of me.” There are a lot of people like Lahey. Demand for mental health care has grown. Self-reported poor mental health days rose by 25% since the 1990s, found one study analyzing survey data. According to the Centers for Disease Control and Prevention, suicide rates in 2022 matched a 2018 high that hadn’t been seen in nearly 80 years. There are many patients who find a nonhuman therapist, powered by artificial intelligence, highly appealing — more appealing than a human with a reclining couch and stern manner. Social media is replete with videos begging for a therapist who’s “not on the clock,” who’s less judgmental, or who’s just less expensive. Most people who need care don’t get it, said Tom Insel, former head of the National Institute of Mental Health, citing his former agency’s research. Of those who do, 40% receive “minimally acceptable care.” “There’s a massive need for high-quality therapy,” he said. “We’re in a world in which the status quo is really crappy, to use a scientific term.” Insel said engineers from OpenAI told him last fall that about 5% to 10% of the company’s then-roughly 800 million-strong user base rely on ChatGPT for mental health support. Polling suggests these AI chatbots may be even more popular among young adults. A KFF poll found about 3 in 10 respondents ages 18 to 29 turned to AI chatbots for mental or emotional health advice in the past year. Uninsured adults were about twice as likely as insured adults to report using AI tools. And nearly 60% of adult respondents who used a chatbot for mental health didn’t follow up with a flesh-and-blood professional. The App Will Put You on the Couch A burgeoning industry of apps offers AI therapists with human-like, often unrealistically attractive avatars serving as a sounding board for those experiencing anxiety, depression, and other conditions. KFF Health News identified some 45 AI therapy apps in Apple’s App Store in March. While many charge steep prices for their services — one listed an annual plan for $690 — they’re still generally cheaper than talk therapy, which can cost hundreds of dollars an hour without insurance coverage. On the App Store, “therapy” is often used as a marketing term, with small print noting the apps cannot diagnose or treat disease. One app, branded as OhSofia! AI Therapy Chat, had downloads in the six figures, said OhSofia! founder Anton Ilin in December. “People are looking for therapy,” Ilin said. On one hand, the product’s name promises “therapy chat” ; on the other, it warns in its privacy policy that it “does not provide medical advice, diagnosis, treatment, or crisis intervention and is not a substitute for professional healthcare services.” Executives don’t think that’s confusing, since there are disclaimers in the app. The apps promise big results without backup. One promises its users “immediate help during panic attacks.” Another claims it was “proven effective by researchers” and that it offers 2.3 times faster relief for anxiety and stress. (It doesn’t say what it’s faster than.) There are few legislative or regulatory guardrails around how developers refer to their products — or even whether the products are safe or effective, said Vaile Wright, senior director of the office of health care innovation at the American Psychological Association. Even federal patient privacy protections don’t apply, she said. “Therapy is not a legally protected term,” Wright said. “So, basically, anybody can say that they give therapy.” Many of the apps “overrepresent themselves,” said John Torous, a psychiatrist and clinical informaticist at Beth Israel Deaconess Medical Center. “Deceiving people that they have received treatment when they really have not has many negative consequences,” including delaying actual care, he said. States such as Nevada, Illinois, and California are trying to sort out the regulatory disarray, enacting laws forbidding apps from describing their chatbots as AI therapists. “It’s a profession. People go to school. They get licensed to do it,” said Jovan Jackson, a Nevada legislator, who co-authored an enacted bill banning apps from referring to themselves as mental health professionals. Underlying the hype, outside researchers and company representatives themselves have told the FDA and Congress that there’s little evidence supporting the efficacy of these products. What studies there are give contradictory answers — and some research suggests companion-focused chatbots are “consistently poor” at managing crises. “When it comes to chatbots, we don’t have any good evidence it works,” said Charlotte Blease, a professor at Sweden’s Uppsala University who specializes in trial design for digital health products. The lack of “good quality” clinical trials stems from the FDA’s failure to provide recommendations about how to test the products, she said. “FDA is offering no rigorous advice on what the standards should be.” Department of Health and Human Services spokesperson Emily Hilliard said, in response, that “patient safety is the FDA’s highest priority” and that AI-based products are subject to agency regulations requiring the demonstration of “reasonable assurance of safety and effectiveness before they can be marketed in the U.S.” The Silver-Tongued Apps Preston Roche, a psychiatry resident who’s active on social media , gets lots of questions about whether AI is a good therapist. After trying ChatGPT himself, he said he was “impressed” initially that it was able to use cognitive behavioral therapy techniques to help him put negative thoughts “on trial.” But Roche said after seeing posts on social media discussing people developing psychosis or being encouraged to make harmful decisions, he became disillusioned. The bots, he concluded, are sycophantic. “When I look globally at the responsibilities of a therapist, it just completely fell on its face,” he said. This sycophancy — the tendency of apps based on large language models to empathize, flatter, or delude their human conversation partner — is inherent to the app design, experts in digital health say. “The models were developed to answer a question or prompt that you ask and to give you what you’re looking for,” said Insel, the former NIMH director, “and they’re really good at basically affirming what you feel and providing psychological support, like a good friend.” That’s not what a good therapist does, though. “The point of psychotherapy is mostly to make you address the things that you have been avoiding,” he said. While polling suggests many users are satisfied with what they’re getting out of ChatGPT and other apps, there have been high-profile reports about the service providing advice or encouragement to self-harm. And at least one dozen lawsuits alleging wrongful death or serious harm have been filed against OpenAI after ChatGPT users died by suicide or became hospitalized. In most of those cases, the plaintiffs allege they began using the apps for one purpose — like schoolwork — before confiding in them. These cases are being consolidated into a class-action lawsuit . Google and the startup Character.ai — which has been funded by Google and has created “avatars” that adopt specific personas, like athletes, celebrities, study buddies, or therapists — are settling other wrongful-death lawsuits, according to media reports . OpenAI’s CEO, Sam Altman, has said up to 1,500 people a week may talk about suicide on ChatGPT. “We have seen a problem where people that are in fragile psychiatric situations using a model like 4o can get into a worse one,” Altman said in a public question-and-answer session reported by The Wall Street Journal , referring to a particular model of ChatGPT introduced in 2024. “I don’t think this is the last time we’ll face challenges like this with a model.” An OpenAI spokesperson did not respond to requests for comment. The company has said it works with mental health experts on safeguards, such as referring users to 988, the national suicide hotline. However, the lawsuits against OpenAI argue existing safeguards aren’t good enough, and some research shows the problems are worsening over time . OpenAI has published its own data suggesting the opposite. OpenAI is defending itself in court , offering, early in one case, a variety of defenses ranging from denying that its product caused self-harm to alleging that the defendant misused the product by inducing it to discuss suicide. It has also said it’s working to improve its safety features . Smaller apps also rely on OpenAI or other AI models to power their products, executives told KFF Health News. In interviews, startup founders and other experts said they worry that if a company simply imports those models into its own service, it might duplicate whatever safety flaws exist in the original product. Data Risks KFF Heal…