Crisis Text Line joins Common Sense Media to set youth AI safety standards

The partnership will focus on how AI products respond to self-harm, suicidality and emotional manipulation involving young people

Crisis Text Line will help the Youth AI Safety Institute develop standards for severe mental health risks in AI products

Crisis Text Line has joined Common Sense Media’s Youth AI Safety Institute as a founding standards partner, bringing evidence from crisis conversations into the development of safety criteria for AI products used by young people.

The partnership will concentrate on three severe risks: self-harm, suicidality and emotional manipulation. 

A safety system that reacts only to a narrow set of high-risk words may miss conversations in which distress develops gradually or appears through changing language.

 Common Sense Media says Crisis Text Line’s experience will inform how products are assessed when a young person expresses distress indirectly, ambiguously or across an extended conversation.

Crisis Text Line provides free, confidential, text-based crisis support in English and Spanish, 24 hours a day. It has trained more than 55,000 volunteers since launching in 2013.

Crisis conversations will inform AI assessments

The Youth AI Safety Institute researches the effects of AI on children, evaluates products and develops standards addressing risks and harms. It says its findings are intended to help technology companies improve product safety and families make informed choices.

Crisis Text Line’s role will be to contribute knowledge drawn from real interactions with people seeking mental health support. 

Robbie Torney, Head of AI and Digital Assessments at the Youth AI Safety Institute, said on LinkedIn: “Crisis Text Line has learned from millions of crisis conversations how young people signal distress. That experience will help us measure AI products against how kids really behave when they’re at risk.”

The intended benchmark goes beyond whether a chatbot can identify explicit statements of self-harm. Common Sense Media argues that AI systems must respond safely when signs of risk are less direct, although the organization has not yet specified what an acceptable response should involve.

Human crisis support remains separate from AI

Crisis Text Line connects people with live, trained volunteer Crisis Counselors rather than automated support. Counselors use active listening, referrals and collaborative problem-solving to help users develop a plan to remain safe.

Supervisors overseeing the counselors hold relevant master’s degrees or comparable crisis intervention experience. If a person is judged to be at imminent risk of suicide or homicide and cannot plan for their own safety, a supervisor may contact emergency services.

In the United States and Puerto Rico, people seeking confidential support can text HOME or HOLA to 741741. The service is also available through web chat and WhatsApp. In the United Kingdom, its international partner Shout can be reached by texting SHOUT to 85258.