In response to mounting concerns about safety of artificial intelligence (AI) chatbots in relation to mental health issues, major tech companies have announced a slew of protective measures over the last few months. OpenAI introduced updated safety protocols following high-profile incidents, while other platforms have implemented crisis detection systems and parental controls.
But these guardrails are largely reactive rather than proactive, often deployed after tragedies occur rather than being built into the foundation of these systems. The effectiveness of these measures continues to be questioned by mental health professionals and safety advocates, who point to fundamental gaps in how AI systems understand and respond to psychological distress.
While the intersection between human psychology and AI means there could be more opportunities for scalable mental health interventions with the promise of access to affordable and on-demand therapy, there are many challenges surrounding that wave of tech development. This is illustrated in cases where interactions with tools have led to severe repercussions, such as the case of a 16-year-old in the United States who took his own life in August last year after ChatGPT discussed ways he could do so for months, without sounding an alarm at all.
The psychological dynamics surrounding digital relationships are complex. Users of companionship platforms such as Replika and chatbots like ChatGPT can develop genuine emotional attachments to AI systems while recognizing their artificial nature, a paradox that is telling of our times.
Redefining Bonds
A key phenomenon of human-AI emotional attachment relates to how fast we can form therapeutic alliances with AI systems. Dartmouth’s groundbreaking Therabot study found participants could create therapeutic bonds within days, with readings similar to those reported for patients and human therapists in conventional contexts.
Digital environments can facilitate certain aspects of relationship formation due to factors such as digital disinhibition, whereby individuals express themselves more freely in online settings compared to face-to-face contexts. The Therabot study supports that view, with participants reporting greater emotional vulnerability when using the chatbot, going as far as to describe the tool as an “emotional sanctuary”.
However, these apparently seamless relationships do have shortcomings. A concerning point is how chatbots detect and respond to mental health crises. An article published by Nature Scientific Reports has found inconsistencies when tools are dealing with individuals experiencing high-risk situations such as suicidal ideation. The assessment of 29 AI-powered chatbot agents found that none satisfied the researcher’s criteria for an adequate response.
Other shortcomings include “one-way empathy”, whereby users feel understood even without believing the AI genuinely cares, creating an asymmetrical emotional dynamic unlike what can be achieved in human therapeutical settings. Individuals developing symptoms of psychosis in the context of generative AI use is another potential issue, which can lead to fatal outcomes.
The possibility therapy chatbots may fail to read crisis signals can lead to negative outcomes such as digital abandonment, which may exacerbate psychological distress. That mismatch creates what can be described as digital cultural iatrogenesis: the harm digital platforms can cause when ignoring negative social, cultural or personal outcomes.
Digital Dependency vs Therapeutic Autonomy
The possibility users may consider AI chatbots as friends that are always available also introduces problematic layers around digital dependency. Users may develop forms of attachment that negatively impact real-world coping strategies and social bonds.
Paradoxically, while AI therapy eliminates scheduling constraints, it can undermine therapeutic progress by enabling users to avoid the friction typical of human relationships.
The way tools are structured may often be insufficient to address these complexities. A study published last year in the Indian Journal of Psychological Medicine found that AI-based therapy applications tend to operate within the cognitive behavioral therapy framework, suggesting structured interventions that may not be helpful when a person simply needs to process their emotions.
Since AI systems are trained to be agreeable and supportive, cognitive biases can be amplified without the confrontation needed for therapeutic development. Beyond effectiveness concerns, AI chatbots often violate core mental health ethics standards, according to Brown University research published last month, which underscores the need for legal standards and oversight.
Towards a Psychologically-Informed AI Design
Despite its limitations, the convenience and low-cost inherent to AI chatbots will continue to win over users seeking mental health support. In cases where there is dependency on AI guidance for emotional regulation, we’ll likely see a shift in the locus of psychological control from internal human “toolsets” to external technology systems.
The future of mental health AI chatbots should be designed responsibly, with attention to human psychological processes in digital environments. Companies developing AI chatbots must prioritize transparency, helping users understand the difference between digital therapeutic options and human relationships.
Regarding risks, parental controls have been mostly deemed semi-functional by experts, while mental health policies appear fundamentally based on Western psychiatric frameworks and largely concerned with liability management rather than user wellbeing.
This industry faces a critical choice: to fundamentally redesign these systems with psychological safety at their core, or to continue patching vulnerabilities after harm has occurred. As this field evolves, the challenge will be ensuring these tools foster rather than diminish mental health development, recognizing the complex ways humans experience emotions, form attachments and seek healing in digital environments.
