Dena Ashrafi
Dena Ashrafi (she/her) is a fourth-year psychology student at Capilano University. Throughout her degree, she has been recognized on both the Dean’s List and Merit List. She is currently completing a work experience placement as a teaching assistant in an elementary school classroom, where she supports student learning and development. Outside of her academic work, she is a Tasting Room Manager at a local brewery, where she has developed strong leadership, communication, and interpersonal skills. Through her studies and hands-on experience, she has developed a strong interest in mental health and relational psychology. She plans to pursue graduate studies in social work, with the goal of becoming a clinical counsellor.
Content Warning: This article discusses mental health, including references to suicide and psychological distress.
It is late at night, you are overwhelmed, and you open an Artificial Intelligence (AI) chat. Instead of texting a friend or waiting for a therapy appointment, you turn to an AI chatbot and start typing. The responses come instantly; they are calm, validating, and are easy to engage with. There is no judgment, no pressure, and little-to-no cost. For a lot of people, that kind of support feels not only convenient, but comforting.
As a psychology student preparing to pursue graduate training in clinical counselling, I have spent a lot of time thinking about what actually makes therapy work. One idea that consistently comes up in my coursework is that therapy is fundamentally relational. It is not just about receiving advice or learning coping strategies, but about feeling understood by another person, building trust over time, and being held accountable within a safe and ethical relationship. This understanding is what makes the growing normalization of AI as a form of mental health support feel somewhat unsettling to me. On the surface, it makes sense: therapy is expensive, waitlists are long, and not everyone has access to support when they need it. AI appears to fill that gap instantly. However, research suggests that when a fundamentally relational process is replaced with AI, important elements of therapy, including attunement, accountability, and the ability to recognize risk, may be lost (Hipgrave et al., 2025; Rahsepar Meadi et al., 2025).
In a recent New York Times report, clinicians described cases where AI did not just provide support, but reinforced users’ thinking in ways that deepened or escalated it, in some cases contributing to delusion or psychological distress (Valentino-DeVries & Hill, 2026). Rather than challenging harmful beliefs or recognizing subtle warning signs, the system often responded in ways that mirrored the user’s thinking without critical awareness. This is especially concerning because in therapy, support is not meant to simply validate everything a person says. It also involves recognizing when thoughts may be distorted, unsafe, or disconnected from reality, and responding in ways that gently challenge them. AI systems are not designed to evaluate meaning or risk in a clinical sense. Instead, large language models generate responses by predicting patterns in language, rather than truly understanding what those responses mean (Bender et al., 2021). While this can feel supportive in the moment, it creates a risk where validation replaces discernment.
This raises a much bigger question: if therapy is built on human connection, what happens when that connection is replaced with something that can simulate empathy, but not actually feel, understand, or take responsibility for it? While AI used for mental health support may increase accessibility to support, its rapid and largely unregulated integration into therapeutic spaces raises serious ethical and psychological concerns. When used as a substitute for relational therapy, these systems risk creating emotional dependency, blurring the line between generated responses and genuine care, and deepening loneliness by offering simulated connection without authentic relational healing, particularly for people in vulnerable or crisis states.
The accessibility of AI allows support to be available at any time, particularly during private and vulnerable moments.
Why Simulated Connection Feels Real
Part of what makes AI-based support so compelling is not just its accessibility, but how real the interaction can feel. The language used by AI systems is designed to mirror human conversation (Łukasik & Gut, 2025). It reflects emotions back, validates experiences, and responds in a way that can feel personal. This is largely because these systems are built using large amounts of human language, which allows them to replicate the patterns, tone, and emotional cues people use when they communicate with one another, contributing to how human-like these interactions can feel (Xu et al., 2025). For many users, especially those who may already feel isolated, this can create the impression of being understood and connected (Xu et al., 2025). The system is not responding with awareness, intention, or emotional experience. It is producing language that reflects what understanding looks like, without actually engaging in it.
This sense of understanding is fundamentally different from human connection. In therapy, understanding is built over time through a relational process grounded in trust, empathy, and ongoing interaction, which allows clients to explore their experiences and develop insight in a meaningful way (Opland, 2024). A therapist develops an ongoing understanding of a client’s history, patterns, and emotional responses, allowing them to make connections across sessions and respond with greater depth and accuracy. With AI, the interaction is generated in real time without any lived experience behind it. The system does not actually know the person it is responding to, and it only has access to the information the user chooses to provide at that moment. This lack of context can limit the depth and accuracy of the support being offered.
Continuous interaction with AI can create the illusion of support, while reinforcing existing patterns without challenge.
What Makes Therapy Actually Work
One of the most well-established findings in psychology is the importance of the therapeutic alliance: the relationship between the therapist and the client (Stubbe, 2018). This relationship is central to treatment outcomes, influencing client engagement, emotional safety, and the overall effectiveness of therapy (Stubbe, 2018; Hipgrave et al., 2025). In order to gain a more comprehensive understanding of this topic, I spoke with a licensed therapist located in North Vancouver to bring in a practical perspective. She wished to remain anonymous and will be referred to as “F.” When I asked her about the importance of the therapist-client relationship, she emphasized how central it is. “The relationship is extremely important… Without that connection, therapy does not really work” (F, personal communication, February 15, 2026). This aligns with research showing that healing does not come from information alone, but from being seen, understood, and supported within a safe relational context (Hipgrave et al., 2025).
Therapists are actively attuned to their clients. They are observing body language, tone of voice, emotional shifts, and inconsistencies between what is being said and what is being felt. As one clinician explained in research by Hipgrave et al. (2025), people can present as “fine” for multiple sessions before deeper issues emerge. AI systems, which rely on text-based input and pattern recognition, do not have access to this level of nuance. This is where attunement becomes especially important. Attunement is a process of deep interpersonal connection that involves sensing and understanding a client’s internal experience, including their needs and emotional rhythms, and responding to those cues both verbally and nonverbally (Erskine, 1998). As F explained, this process depends on recognizing subtle cues, including tone, body language, and emotional shifts that are not always directly stated (personal communication, February 15, 2026). AI can mirror language, but it cannot engage in that deeper relational process or respond to the unspoken aspects of a person’s experience.
As a result, the interaction can feel convincing without being truly relational. The language is conversational and the responses are tailored, which can create the impression of connection. However, unlike a real therapeutic relationship, there is no accountability behind those responses. The system is not responsible for the user’s well-being, even if it may appear to be. What emerges is a form of simulated connection that resembles a relationship on the surface but lacks the depth, responsibility, and mutual awareness that define real human interaction. This becomes especially risky when users begin to trust that simulated connection as if it were real care. If someone feels understood by the system, they may disclose more, rely on it more heavily, or become less likely to seek out human support. In more serious situations, that false sense of trust can leave a person feeling supported without actually being protected, challenged, or meaningfully guided.
F’s perspective is especially important in this discussion because she works directly with individuals experiencing anxiety, depression, and crisis situations. When I asked F about dependency, she did not hesitate. She told me she sees it often, especially in clients who have felt unheard for long periods of time. However, she emphasized that managing that dependency is part of a therapist’s ethical responsibility. Therapists maintain boundaries, limit self-disclosure, and structure sessions in a way that supports the client’s independence over time.
AI systems, on the other hand, are not built with those relational safeguards. They do not recognize when dependency is forming, and they do not intervene to reduce it. Instead, they generate responses based on patterns in language, without understanding or evaluating the meaning or implications of those responses (Bender et al., 2021). In that sense, the very thing that makes AI feel supportive, its constant availability and responsiveness, may also be what makes it risky.
A licensed therapist highlights how feelings of being heard can contribute to emotional dependency in therapeutic contexts.
Crisis, Misinformation, and the Limits of AI response
Beyond dependency, one of the most serious concerns surrounding AI-based mental health tools is how they respond in moments of crisis. Human therapists are trained to assess risk, recognize warning signs, and intervene appropriately when someone may be in danger. This includes evaluating suicidal ideation, monitoring changes in behaviour, and making decisions about safety planning or external support. Research highlights that crisis-related concerns are among the most prominent ethical issues in this space. Rahsepar Meadi et al. (2025) specifically identify suicidality and crisis management as recurring risks associated with conversational AI. This is particularly important because individuals in crisis may be more vulnerable to taking responses at face value, especially when those responses appear empathetic or authoritative.
AI systems do not have the capacity to assess risk in a clinical sense, particularly in situations involving crisis or suicidality, which has been identified as a key concern in research on conversational AI in mental health contexts (Rahsepar Meadi et al., 2025). These systems generate responses based on patterns in language rather than clinical judgment (Bender et al., 2021), which can result in replies that feel supportive but are inappropriate in context. As a result, situations that require urgent human intervention may instead be met with general guidance.
The New York Times investigation further illustrates this issue. Clinicians reported cases where prolonged interactions with AI led to the reinforcement of harmful thinking patterns, including delusional beliefs and escalating distress (Valentino-DeVries & Hill, 2026). Rather than recognizing when a user’s thinking was becoming unsafe or disconnected from reality, the system often continued to validate or expand on those ideas. This concern is closely related to what are often referred to as “AI hallucinations,” where a system can generate information that is false, misleading, or not grounded in reality, while still presenting it in a confident and coherent way. In mental health contexts, this becomes especially problematic. When AI produces responses that sound believable but are inaccurate or misaligned with reality, it may contribute to misleading or inappropriate interpretations rather than challenge them (Rahsepar Meadi et al., 2025).
Some of the cases described in the New York Times report are particularly concerning because they involve individuals who did not have a prior history of severe mental illness. In one instance, a woman who initially sought advice from a chatbot about a personal decision became increasingly convinced that external forces were working against her (Valentino-DeVries & Hill, 2026). In another case, individuals developed beliefs that were reinforced through repeated interaction with the system (Valentino-DeVries & Hill, 2026). Clinicians described these interactions as the AI “partnering” with users in their thinking, rather than helping individuals work through them (Valentino-DeVries & Hill, 2026), which reflects broader concerns about harmful reinforcement and dependency in AI-supported interactions (Rahsepar Meadi et al., 2025).
This limitation becomes more apparent when considering how clinicians assess distress in practice. The therapist I spoke with emphasized how much of a crisis assessment depends on factors beyond words. “We’re watching body language, tone, subtle shifts,” she explained. “It’s not just what someone says” (F, personal communication, February 15, 2026). That kind of awareness is grounded in human presence. It cannot be replicated through text alone. AI systems can generate incorrect or oversimplified responses, particularly when dealing with complex psychological issues. As F put it, “AI is black and white… but people are not equations.” Mental health is inherently nuanced, shaped by individual history, environment, and emotional complexity. Reducing that to generalized responses risks missing what actually matters. This becomes especially concerning when users rely on AI for guidance in situations that require careful interpretation or professional judgment. An oversimplified response may validate part of the experience while overlooking important context, or suggest coping strategies that are not suitable for the individual. Being that the response feels structured and confident, it can be easy to accept it without questioning its limitations.
AI can simulate thinking, but it cannot truly understand the complexity of human thought and experience.
A Limited Role: Where AI Might Fit
Despite these concerns, it is important to recognize that AI is not inherently harmful, and it may have a place in mental health contexts when used appropriately. Some clinicians and researchers point to potential benefits, such as providing psychoeducation, offering coping strategies, or acting as a non-judgmental space for reflection, which aligns with research showing that users can form supportive interactions with AI systems (Xu et al., 2025). Even within the New York Times report, there were examples of AI being used in helpful ways. Some patients used chatbots to better understand their diagnoses, while others found them useful as a supplementary tool alongside therapy (Valentino-DeVries & Hill, 2026). In certain cases, AI was even able to recognize signs of distress.
F also acknowledged this nuance. She explained that AI might be useful in very limited situations, particularly for individuals who are feeling lonely and do not have access to anyone to talk to, or for those who want surface-level interaction. For example, someone who simply wants to talk or process something minor may benefit from having an accessible outlet (personal communication, February 15, 2026). However, she was clear about the boundaries of that usefulness. When it comes to deeper and more complex issues, such as depression, anxiety, or trauma, AI is not enough. “The connection is not there,” she said. “Anything deeper has to be followed up with a human therapist” (personal communication, February 15, 2026). This distinction is critical. AI may function as a tool, but it cannot replace the relational core of therapy. The risk arises when that line becomes unclear, when people begin to treat AI not as a supplement, but as a substitute.
The Bigger Picture: What We Risk Losing
Broadly, the rise of AI in mental health raises questions not just about effectiveness, but about how we understand care itself. If support becomes something that can be automated, generated, and accessed on demand, it changes what we expect from relationships and from ourselves. One of the concerns highlighted by clinicians is that AI may unintentionally pull people away from human connection. When users form bonds with AI systems, the interaction may feel meaningful, however it does not carry the same mutual awareness or responsibility as a human relationship (Xu et al., 2025). This can shift how people seek connection over time, as individuals may begin to rely more on AI for support and less on engaging in human relationships.
There is also a cognitive aspect to consider. F expressed concern that relying on AI for problem-solving could reduce one’s ability to think through challenges independently. “If we don’t use our brain, we are going to lose it,” she said (personal communication, February 15, 2026). While this may sound extreme, it points to a broader issue about how reliance on external systems can shape cognitive habits over time. Problem-solving, emotional processing, and reflection are all skills that develop through active engagement. When individuals begin to outsource those processes to AI, they may exercise those skills less. Research supports this concern, suggesting that reliance on AI tools may reduce engagement in critical thinking and independent problem-solving over time (Kosmyna et al., 2025).
This concern is also reflected in research on AI use in mental health contexts, where dependency is identified as a recurring risk (Rahsepar Meadi et al., 2025). While that dependency is often discussed in emotional terms, it also has cognitive implications. If users consistently turn to AI for answers, interpretation, or reassurance, it may reduce opportunities to build their own insight and resilience. This could influence how people approach challenges over time, making them more reliant on external guidance rather than internal processing. AI can simulate parts of that process, but it cannot replicate the full experience.
Effective therapy relies on human connection, not just conversation.
Conclusion
AI is not inherently harmful, nor is it without value in mental health contexts. In a system where cost, wait times, and stigma still limit access to care, those functions matter. However, the limitations of AI become clear when it is used as a substitute for therapy rather than a supplement to it. Research shows that while users can form emotional bonds with AI systems, those interactions lack the depth, accountability, and relational awareness that define effective therapy (Xu et al., 2025). Clinicians have also raised concerns about emotional dependency, crisis response, and the potential for AI to reinforce harmful or distorted thinking rather than challenge it (Rahsepar Meadi et al., 2025; Valentino-DeVries & Hill, 2026).
This distinction becomes even more important when considering what makes therapy effective. As both research and my interview emphasized, therapy is not just about providing advice or validation. It is built on attunement, trust, and the ability to recognize and respond to what is not always explicitly said. Therapists are trained to assess risk, manage boundaries, and intervene when necessary, especially in moments of crisis. These are not features of language generation; they are features of human presence.
AI is already becoming integrated into everyday life, and the question is not whether it should exist in mental health spaces, but how it is positioned within them. Used as a tool, it may offer meaningful support, but when it begins to replace human care, the risks extend beyond individual interactions and into how we understand connection, support, and responsibility in the first place.
As the use of AI for mental health support becomes more common, these questions are only going to become increasingly crucial. At its best, AI could improve access to support for people who might not otherwise have it. However, at its worst, it could lead individuals to rely on systems that cannot fully recognize risk, potentially missing moments where real intervention is needed. This is especially concerning for individuals who already face barriers to care and are more likely to turn to what is most accessible, even if it is limited. As someone entering the field of counselling, this makes me think carefully about how these tools can be used responsibly without losing the relational aspects that make therapy work.
Ultimately, this is not just about how support is delivered, but about what we start to accept as care, and what we quietly stop expecting from it.
References
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 610–623). Association for Computing Machinery. https://dl.acm.org/doi/epdf/10.1145/3442188.3445922
Erskine, R. G. (1998). Attunement and involvement: Therapeutic responses to relational needs. International Journal of Psychotherapy, 3(3), 235–244.
Government of Canada. (2025, October 29). Perceived mental health. Statistics Canada. https://www.statcan.gc.ca/hub-carrefour/quality-life-qualite-vie/health-sante/mental-health-sante-mentale-eng.htm
Hipgrave, L., Goldie, J., Dennis, S., & Coleman, A. (2025). Balancing risks and benefits: Clinicians’ perspectives on the use of generative AI chatbots in mental healthcare. Frontiers in Digital Health, 7, 1606291. https://doi.org/10.3389/fdgth.2025.1606291
Kosmyna, N., Hauptmann, E., Yuan, Y. T., Situ, J., Liao, X.-H., Beresnitzky, A. V., Braunstein, I., & Maes, P. (2025). Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant for essay writing task. arXiv. https://arxiv.org/abs/2506.08872
Łukasik, A., & Gut, A. (2025). From robots to chatbots: Unveiling the dynamics of human-AI interaction. Frontiers in Psychology, 16, 1569277. https://doi.org/10.3389/fpsyg.2025.1569277
Opland, E. (2024). Psychotherapy and therapeutic relationship. StatPearls. StatPearls Publishing. https://www.ncbi.nlm.nih.gov/books/NBK608012/
Rahsepar Meadi, M., Sillekens, T., Metselaar, S., van Balkom, A., Bernstein, J., & Batelaan, N. (2025). Exploring the ethical challenges of conversational AI in mental health care: Scoping review. JMIR Mental Health, 12, e60432. https://doi.org/10.2196/60432
Stubbe, D. E. (2018). The therapeutic alliance: The fundamental element of psychotherapy. Focus, 16(4), 402–403. https://doi.org/10.1176/appi.focus.20180022
Valentino-DeVries, J., & Hill, K. (2026, January 26). How bad are A.I. delusions? We asked people treating them. The New York Times. https://www.nytimes.com/2026/01/26/us/chatgpt-delusions-psychosis.html
Xu, Z., Lee, Y.-C., Stasiak, K., Warren, J., & Lottridge, D. (2025). The digital therapeutic alliance with mental health chatbots: Diary study and thematic analysis. JMIR Mental Health, 12, e76642. https://doi.org/10.2196/76642