There is a particular kind of message that arrives when someone is not coping. It might be an email to an Employee Assistance Program, a call booked after a sleepless week, or a quiet click on a page titled "Getting support". The person reading it is rarely at their best. They may be grieving, frightened, exhausted or ashamed. Every word they meet in that moment carries more weight than it would on an ordinary day.
That is the setting I keep returning to when people ask me how AI should be used in mental health support. Not the demo, not the efficiency case, but the person on the other side of the screen who is trying to hold themselves together.
I work with AI every day. I build with it, I advise organisations on it, and I believe it can widen access to help in ways that matter. That is exactly why I think the boundaries around it need to be drawn with more care here than almost anywhere else.
Words that land on people who are struggling
Most AI conversations in organisations are about speed. Draft the policy faster, summarise the report, produce the training module in an afternoon instead of a month. In many fields that is a reasonable trade. A slightly generic sales email costs very little.
In wellbeing work the cost is different. A message that is meant to reassure can feel cold when it is written with no sense of who will read it. A training scenario about responding to distress can teach the wrong instinct if nobody who has actually sat with distress has shaped it. A phrase that seems neutral on a screen can sound dismissive to someone who has just lost a parent, a job or a marriage.
AI is very good at producing language that sounds right. It is far less reliable at knowing when sounding right is not the same as being right. It does not carry the memory of a conversation that went badly because of one careless sentence. It has never watched someone's shoulders drop when they finally felt heard. Those experiences are what make good support material good, and they cannot be generated.
So the first principle is simple. When the audience is people in difficult, stressful or disrupted situations, AI can assist with the writing, but it should never be the only author.
What ethical use actually looks like
Ethical AI in this space is not a statement on a website. It is a set of decisions made in the work, again and again, by people who understand what is at stake.
It starts with being clear about where AI belongs. It can help a clinician or coach organise their thinking, test a draft for clarity, adapt reading level, translate material so it reaches more communities, or spot gaps in a training programme. Used this way, it gives skilled people more time for the parts of the work only they can do.
It also means being clear about where it does not belong. AI should not be deciding what a person in crisis needs to hear. It should not be the unreviewed voice of an organisation's care. It should not quietly replace the practitioner whose judgement is the reason the service exists.
In practice that looks like a few firm habits:
- Every piece of support material is reviewed by someone with real clinical or coaching experience before it reaches the people it is meant to help.
- Training that guides how people respond to distress is designed by practitioners first, with AI used to refine rather than to originate.
- People are told honestly when they are interacting with an automated system, and they always have a clear path to a human.
- Personal and sensitive information is treated with the restraint it deserves, and never fed into tools simply because it is convenient.
- Someone is accountable for the words, by name, not "the system".
None of this is complicated. What makes it hard is that each shortcut looks harmless on its own. One module written entirely by a model, one reassuring message sent without review, one chatbot answering questions it was never built to hold. The drift happens quietly, and by the time anyone notices, the organisation's care has started to sound like nobody in particular.
Support material is not content. It is a promise that someone thought carefully about the person reading it.
Why the coach is still the core
There is a deeper reason I am cautious, and it has to do with how people actually grow.
My coaching training taught me something I have seen confirmed many times since. People rarely change because they were given the right information. They change because someone stayed with them while they found their own way to it. The question asked at the right moment. The silence that was allowed to last a little longer. The reflection that showed them a pattern they had not been able to see alone.
That is relational work. It depends on presence, attunement and trust built over time. A coach notices the hesitation before an answer, the topic that keeps being avoided, the energy that shifts when a particular person is mentioned. A coach can hold a hard truth gently enough that it can be heard. These are not features waiting for a better model. They are what happens when one human being pays full attention to another.
AI can support that relationship. It can help a coach prepare, keep notes in order, or offer a client a structured exercise between sessions. It can make good coaching easier to reach for people who would otherwise go without. But it cannot be the relationship. When we let it stand in for one, we teach people that being helped means being processed, and that is a lesson that does lasting damage.
The same is true of training. The managers, peer supporters and frontline staff who learn to respond to distress are learning a human skill. They need to hear from people who have done it, who have made mistakes and learned from them, who can describe what it actually feels like to sit across from someone who is falling apart. A training module that has never been touched by that experience will teach the vocabulary of care without its substance.
Keeping control of what matters
Controlling AI usage in mental health support is not about fear of the technology. It is about knowing which parts of the work are made of information and which parts are made of people.
The organisations that get this right will use AI generously in the background. They will free their practitioners from administration, make their materials clearer and more accessible, and reach people earlier. And they will protect, deliberately and visibly, the moments where a human voice is the whole point.
Those moments are not inefficiencies to be engineered away. They are the reason someone picked up the phone in the first place.