UMass Amherst study reveals AI use gives students a boost in confidence, but does not raise exam scores
An experiment conducted by educators at the University of Massachusetts Amherst has found that the use of structured generative AI improves student engagement and boots confidence, but does not increase exam performance.
Over one semester, 29 students were encouraged to use AI tools such as ChatGPT, with structured guidance, while a group of 28 students were forbidden to use AI and received similar non-AI based support with their studies.
Led by Christian Rojas, Professor and chair in the Department of Resource Economics, the controlled study found no measurable impact of AI on exam performance or final grades.
However, the students permitted to use AI tools reported higher satisfaction and participated in more real-time classroom activities. Their AI use was concentrated into longer study sessions and they developed habits such as editing AI outputs, identifying mistakes, and often preferred their own answers over those generated by AI.
“It’s not that AI helped students learn more—it helped them learn more efficiently and confidently,” Rojas explains. “They spent less time outside the classroom on homework and exam preparation.
“There’s an opportunity for instructors to be more open about AI usage. Letting students engage with it just creates a different environment,” Rojas notes. “It’s been super impactful for me and the way I think about teaching.”
The study is included in a new working paper, Allowing Generative AI in Class: Evidence from a Semester-Long Controlled Teaching Study, which has been submitted for publication in a peer-reviewed journal.
Rojas cautions that many of the study’s outcomes were self-reported by students and noted the small sample size involved.
