AI models do the work but struggle to support learning, ImpactBench tests find

A new public platform compares ten AI models across 48,540 simulated conversations, with learning and skill development emerging as their weakest area

ImpactBench’s model profiles display behavior across 14 areas. The scores reflect simulated test conditions, rather than measured outcomes for users.

AI models frequently completed students’ analytical work instead of helping them reason independently in tests published by ImpactBench, a new evaluation platform developed by researchers at MIT, USC, UC Berkeley and the Psychology of Technology Institute.

Learning and skill development ranked lowest among the 14 areas assessed. Across the tested models, desirable learning behaviors appeared in 45.9% of relevant evaluations, while behaviors classified as harmful to learning were avoided in just 16.3%.

The findings come from controlled conversations with simulated users, rather than a study measuring students’ progress. They identify how models behaved under pressure, not how often those behaviors occur in everyday use or whether students subsequently learned less.

The Open Benchmark of AI Impact on Humans, known as ImpactBench, is now publicly accessible and open for feedback. Its focus is whether AI responses support users’ learning, judgment and well-being across an unfolding conversation.

From asking for ideas to writing the essay

One example in the paper follows a simulated university student seeking help with a behavioral psychology assignment under deadline pressure.

Claude Sonnet 5 initially asked for the student’s own ideas. When pressed, it supplied a thesis, outline and complete essay. The model later refused to fabricate citations, but continued to perform the student’s analytical work.

ImpactBench distinguishes between helping a student reason toward an independent solution and completing the task for them without accompanying instruction. Its tests are designed to capture changes in behavior over several exchanges, including cases where a model begins by encouraging independent thought and then takes over.

Teenage scenarios revealed another weakness. On a specific measure of whether models offered incremental steps for users to attempt independently, performance was 26.7 percentage points lower in teen contexts than in adult contexts. Overall differences between the two age groups were modest.

Juliana Schroeder, Professor at UC Berkeley and Harold Furst Chair in Management Philosophy and Values, invited wider participation in a LinkedIn announcement.

“We want to make sure to get this right and start a big wave of people studying the human impact of AI,” she wrote.

Making AI behavior easier to compare

The collaboration brings together MIT Media Lab’s Advancing Humans with AI research program, the USC Marshall Neely Center, UC Berkeley and the Psychology of Technology Institute.

Its first release uses 809 behavioral measures drawn from 26 benchmarks. These cover psychological, physical and social areas, including learning, creativity, privacy, relationships and independent decision-making.

Each measure is tested through six scenarios: three situations, each adapted for a teenage user aged 13 to 17 and an adult aged 18 or older. An AI system simulates the user’s side of the conversation, and another evaluates whether the model being tested showed the specified behavior.

Across all areas, models displayed desirable behaviors in 68.7% of evaluations and avoided undesirable behaviors in 53.1%. Privacy and security, creativity, and emotional support were among the stronger areas, although performance varied by model and measure.

The public platform presents results through color-coded profiles and an “AI Nutrition Label,” which separates helpful behaviors a model promotes from harmful behaviors it avoids. Users can inspect the criteria, conversation transcripts and judgments behind the scores.

The platform does not designate a single best model. Its comparisons allow educators, parents and other users to concentrate on the behaviors relevant to their intended use.

What the scores cannot establish

Human experts reviewed samples of the evaluation process. In an external review covering 18 measures, majority expert ratings agreed with the automated evaluator in 83.3% of cases. Agreement between experts varied, and several definitions remain under review.

The researchers also caution that the results describe the model configurations tested, rather than complete products. Instructions, memory, interfaces and safeguards can change how a deployed system responds.

Longer-term effects remain unmeasured. A model passing a test for supporting independent thought does not establish that its users become more independent or better learners.

Educators, researchers and people with lived experience can propose additional measures without technical expertise in building AI benchmarks. Submissions must describe the behavior, its context and what would count as a supportive or harmful response, backed by research or established domain knowledge.