OpenAI adds ChatGPT abuse warnings and free child-safety detection tool

Vetted organizations will receive access to an image classifier, while new ChatGPT interventions remove flagged conversations and suspend access for 24 hours

OpenAI’s new ChatGPT intervention removes flagged conversations, suspends access for 24 hours and directs users toward specialist support.

OpenAI is introducing warnings and temporary account suspensions in ChatGPT alongside a free image classifier designed to help safety teams detect potential child sexual abuse material (CSAM), including previously unidentified and AI-generated images.

The announcement also includes developer guidance and tools for specialist law enforcement investigators. Access to the classifier is restricted to vetted organizations through the OpenAI API, while investigative tools will require agency authorization and approval under the program’s requirements.

ChatGPT warnings interrupt harmful behavior

The new ChatGPT intervention interrupts attempts to find abusive material, explains the harm involved and directs users toward specialist support. OpenAI says it will remove the flagged conversation and suspend access for 24 hours, keeping support resources available during that period.

According to the company, repeated violations may result in longer restrictions, account closure or reports to authorities. The intervention builds on existing policies prohibiting CSAM and safeguards designed to refuse requests for it.

Organizations and academic advisers involved in developing the approach include Protect Children, the Lucy Faithfull Foundation, Dr. Caoilte Ó Ciardha, Researcher & Honorary Reader in Forensic Psychology at the University of Kent, and Joel Scanlan at the University of Tasmania. Ó Ciardha and Scanlan independently evaluated a trial of deterrent messages.

Scanlan, Associate Professor at the CSAM Deterrence Centre, describes how that expertise informed the intervention: "OpenAI has engaged multiple external experts who have previously designed and evaluated warning messages, built the intervention with them through evidence-based co-design, and aims to redirect users to local support options in as many jurisdictions as possible."

Classifier targets previously unidentified images

Developed with assistance from the National Center for Missing & Exploited Children (NCMEC), the classifier looks for visual indicators of potential abuse and returns scores to help reviewers prioritize images. It also provides a separate signal indicating whether suspected material may be AI-generated.

OpenAI’s internal evaluation suggests the system could reduce the volume of images requiring review. In a comparison involving user-uploaded images, the company estimates that the classifier would flag 48% fewer images while identifying roughly the same number of abusive images as its existing tool. Flagged images were about twice as likely to contain abuse.

Those findings come from OpenAI’s evaluation, rather than an independent assessment. Early partners have conducted additional tests: in fal’s evaluation of human-reviewed production images, reviewers confirmed 72% of high-confidence flags as previously unidentified CSAM.

John Shehan, Senior Vice President at NCMEC, says: "This new classifier gives more organizations access to tools that can help detect previously unidentified CSAM, strengthen reporting and provide critical information to those working to identify and protect victims."

OpenAI has engaged the Tech Coalition and ROOST to help share access to the tool. It has also begun participating in the Tech Coalition’s Lantern program, exchanging signals about online child sexual exploitation and abuse across platforms.

Developer guidance and investigator training

New developer guidance covers prevention, user reporting, detection, review processes and reporting suspected abuse. Developed with input from the Tech Coalition, Thorn, NCMEC and the Internet Watch Foundation, it also addresses support for people reviewing sensitive content.

Separately, OpenAI, Anthropic and Google jointly commissioned Tech Coalition guidance on responding when users disclose child sexual exploitation and abuse within AI systems.

For specialist law enforcement units, the first investigative tool is designed to review lengthy chat logs and records returned through legal processes, identify potential connections and create timelines. Findings will be linked to source material.

Initial training is planned as an overview and demonstration, accompanied by a practical field guide. Program administrators will manage access, with further training proposed for regional, national and international conferences.

ETIH Innovation Awards 2027 open for entries

The ETIH Innovation Awards 2027 are now open for entries, with 26 categories recognising companies, institutions, projects and individuals making a measurable impact across education technology. The global awards span AI in education, digital learning, higher education, skills and workforce, inclusion, education operations, evidence and impact, with entries managed through ETIH’s new online awards platform.

Entries are open worldwide, with the early-bird deadline on Friday 18 December 2026 and the final entry deadline on Friday 26 February 2027 at 23:59 UK time. 

Explore the 2027 categories and judging criteria, read the How to Enter guide or start an entry through the ETIH Innovation Awards 2027 platform.

Next
Next

Iowa State secures $6 million USDA grant for ag tech education and workforce expansion