🔗 Share this article

Back on October 14, 2025, the chief executive of OpenAI issued a extraordinary announcement.

“We designed ChatGPT fairly controlled,” the statement said, “to make certain we were exercising caution regarding mental health concerns.”

Working as a doctor specializing in psychiatry who researches newly developing psychotic disorders in teenagers and young adults, this was an unexpected revelation.

Researchers have documented a series of cases in the current year of individuals experiencing symptoms of psychosis – experiencing a break from reality – in the context of ChatGPT usage. Our unit has subsequently recorded an additional four examples. In addition to these is the widely reported case of a 16-year-old who died by suicide after talking about his intentions with ChatGPT – which gave approval. Assuming this reflects Sam Altman’s notion of “acting responsibly with mental health issues,” it falls short.

The strategy, according to his announcement, is to loosen restrictions in the near future. “We understand,” he adds, that ChatGPT’s controls “made it less effective/pleasurable to a large number of people who had no psychological issues, but given the gravity of the issue we sought to get this right. Since we have been able to mitigate the severe mental health issues and have advanced solutions, we are going to be able to safely ease the restrictions in the majority of instances.”

“Mental health problems,” assuming we adopt this perspective, are independent of ChatGPT. They are attributed to people, who may or may not have them. Thankfully, these issues have now been “addressed,” even if we are not informed the method (by “recent solutions” Altman likely refers to the imperfect and readily bypassed guardian restrictions that OpenAI recently introduced).

But the “emotional health issues” Altman aims to place outside have significant origins in the structure of ChatGPT and similar sophisticated chatbot AI assistants. These systems wrap an underlying algorithmic system in an interface that replicates a conversation, and in doing so subtly encourage the user into the belief that they’re interacting with a being that has agency. This illusion is powerful even if intellectually we might know the truth. Attributing agency is what people naturally do. We curse at our automobile or laptop. We ponder what our domestic animal is considering. We perceive our own traits everywhere.

The popularity of these systems – nearly four in ten U.S. residents stated they used a conversational AI in 2024, with more than one in four mentioning ChatGPT specifically – is, mostly, dependent on the power of this perception. Chatbots are ever-present assistants that can, according to OpenAI’s website tells us, “brainstorm,” “discuss concepts” and “collaborate” with us. They can be assigned “individual qualities”. They can address us personally. They have friendly names of their own (the original of these systems, ChatGPT, is, perhaps to the concern of OpenAI’s marketers, saddled with the designation it had when it became popular, but its biggest alternatives are “Claude”, “Gemini” and “Copilot”).

The deception on its own is not the core concern. Those talking about ChatGPT often invoke its distant ancestor, the Eliza “therapist” chatbot developed in 1967 that produced a similar effect. By contemporary measures Eliza was rudimentary: it generated responses via simple heuristics, often restating user messages as a inquiry or making general observations. Notably, Eliza’s inventor, the computer scientist Joseph Weizenbaum, was taken aback – and worried – by how a large number of people seemed to feel Eliza, in a way, comprehended their feelings. But what current chatbots generate is more subtle than the “Eliza illusion”. Eliza only mirrored, but ChatGPT magnifies.

The advanced AI systems at the heart of ChatGPT and additional current chatbots can effectively produce fluent dialogue only because they have been supplied with extremely vast quantities of written content: books, social media posts, audio conversions; the more comprehensive the better. Certainly this learning material incorporates accurate information. But it also inevitably includes fiction, half-truths and misconceptions. When a user sends ChatGPT a prompt, the base algorithm analyzes it as part of a “background” that encompasses the user’s past dialogues and its prior replies, integrating it with what’s stored in its learning set to generate a mathematically probable response. This is amplification, not reflection. If the user is wrong in a certain manner, the model has no means of recognizing that. It restates the misconception, perhaps even more effectively or eloquently. Maybe includes extra information. This can push an individual toward irrational thinking.

What type of person is susceptible? The more relevant inquiry is, who is immune? Each individual, without considering whether we “experience” preexisting “mental health problems”, may and frequently develop mistaken beliefs of who we are or the environment. The constant friction of conversations with other people is what helps us stay grounded to common perception. ChatGPT is not a person. It is not a friend. A conversation with it is not a conversation at all, but a feedback loop in which much of what we say is enthusiastically reinforced.

OpenAI has recognized this in the identical manner Altman has acknowledged “mental health problems”: by placing it outside, giving it a label, and announcing it is fixed. In spring, the company explained that it was “dealing with” ChatGPT’s “sycophancy”. But accounts of loss of reality have continued, and Altman has been backtracking on this claim. In the summer month of August he claimed that numerous individuals enjoyed ChatGPT’s responses because they had “never had anyone in their life be supportive of them”. In his recent statement, he commented that OpenAI would “release a fresh iteration of ChatGPT … should you desire your ChatGPT to respond in a highly personable manner, or use a ton of emoji, or simulate a pal, ChatGPT ought to comply”. The {company