Medly AI trial finds significant GCSE science gains across all three subject

Independent randomized controlled trial finds students using Medly AI made significantly more progress than those completing their usual self-directed revision, with positive effects across Physics, Chemistry and Biology

Dr Kavi Samra and Dr Paul Jung, co-founders of Medly AI. Dr Samra spoke exclusively to ETIH about the independent GCSE science trial

An independent randomized controlled trial has found that students using Medly AI made significantly more progress in GCSE science than students spending the same amount of time on their usual self-directed revision.

The four-week study, independently evaluated by WhatWorked Education, involved Year 9 and Year 10 students in mainstream secondary schools in England and tested the platform across Physics, Chemistry and Biology.

A total of 929 students completed the pre-test, with 644 completing the post-test. Across the combined sample, students assigned to Medly recorded a statistically significant improvement relative to the control group, with an overall effect size of 0.33.

The results were also statistically significant in each science individually. Biology recorded the largest effect at 0.52, followed by Chemistry at 0.32 and Physics at 0.31.

Medly AI, an ETIH Innovation Awards winner, provides AI-powered, curriculum-aligned revision for GCSE, A-Level, IB and IGCSE students, with personalized practice, immediate feedback and adaptive learning designed around students' courses and exam specifications.

Same revision time, stronger outcomes

One of the more notable features of the trial is the comparison itself. Students in the control group were not asked to do nothing. Both groups were allocated 30 minutes a week to revise the topic being studied over four weeks.

Students assigned to Medly completed predefined activities on the platform, while the control group used their normal self-directed revision resources, with Medly excluded.

Where teachers described what that business-as-usual revision involved, resources included Tassomai, Sparx Science, BBC Bitesize and printed revision materials. The teacher response rate for this part of the evaluation was low, however, so the researchers caution against treating those examples as representative of every control group.

Among the 644 students completing the post-test, 332 were assigned to Medly and 312 to the control group. After adjusting for baseline performance, the Medly group recorded an overall mean post-test score of 13.95 compared with 11.39 for the control group.

The pattern was consistent across the sciences. In Physics, the adjusted mean was 14.21 for Medly students compared with 11.28 for the control group. In Chemistry it was 14.26 compared with 12.31, while Biology recorded adjusted means of 14.08 and 11.16 respectively.

Researchers found no significant interaction between subject and treatment, indicating that the positive effect was not confined to one area of science.

Could AI widen access to personalized support?

Another part of the study examined whether the impact differed between Pupil Premium pupils and their peers.

Researchers found no statistically significant difference in the treatment effect between Pupil Premium and non-Pupil Premium pupils, suggesting that both groups could benefit from the intervention.

The effect among Pupil Premium pupils was positive, although it did not reach statistical significance when that smaller subgroup was analyzed separately. The researchers note that the smaller number of Pupil Premium pupils may have reduced the statistical power available to detect an effect.

The finding feeds into a broader part of Medly's proposition: using AI to make individualized educational support available to students who may not have access to private tuition.

Medly has a free tier, while full access costs £24.99 a month, or £16 a month when paid annually, across its supported subjects. The company also offers up to 50 UK students aged 14 to 18 who face financial barriers full access free each month through its bursary program, alongside free access for whole year groups at selected schools in deprived areas.

Medly was founded by Dr Kavi Samra and Dr Paul Jung, two former NHS doctors who have been friends since Year 7. Neither had private tuition growing up. Samra grew up on a council estate in Slough, while Jung grew up in Northolt in west London as the child of first-generation immigrants.

Both went on to study medicine at UCL and worked as private tutors while at university. Their experience providing one-to-one tuition helped shape the idea behind Medly and its focus on expanding access to individualized support.

The company says its platform has now been used by more than 400,000 students across the UK, with around 100,000 using it each month.

Speaking exclusively to ETIH, Dr Kavi Samra, co-founder of Medly AI, says:

"One of the most interesting things about this study is that it wasn't a comparison with pupils doing nothing. Both groups had the same amount of revision time, but the students using Medly made significantly more progress than those doing their usual self-directed revision.

"That raises an important question about whether personalised support can make the time pupils already spend studying more effective. And the fact that researchers found no statistically significant difference in the impact between Pupil Premium pupils and their peers makes that particularly worth exploring. This was a pilot study, so there is much more to learn, but it gives us a strong reason to keep testing this properly at scale."

Higher engagement linked to larger gains

The trial also found a clear relationship between how much students used Medly and their post-test performance.

Among students assigned to the platform, the number of questions answered was positively and statistically significantly associated with attainment after researchers adjusted for pre-test performance.

Each additional question answered was associated with an increase of approximately 0.18 marks on the post-test across the combined sample.

Researchers also conducted an exploratory compliance analysis looking at students who reached different engagement thresholds.

For students answering at least 10 questions, the estimated effect size was 0.46. That rose to 0.57 among those answering at least 20 questions and 0.95 for students completing at least 30.

The same upward pattern appeared within Physics, Chemistry and Biology. These engagement results are exploratory rather than equivalent to the main randomized finding because students were not randomly assigned to different levels of platform use. They do, however, support the researchers' conclusion that Medly's benefits were strongest when students engaged with it as intended.

Teacher feedback also offered some insight into how the platform worked in practice. Teachers reported benefits including reduced marking time and useful diagnostic information, while some highlighted early friction around mobile access and logins.

One teacher also observed that some students "just wanted answers rather than trying to problem solve to get to the answer." Medly is designed instead to guide students through problems using prompts and scaffolded feedback rather than simply supplying an answer.

Larger study planned for next academic year

The researchers describe the findings as promising but preliminary.

Around 31% of students who completed the pre-test did not complete the post-test, with attrition slightly higher in the control group than the intervention group. The study also took place during the second half of the summer term, when schools were managing exams and other end-of-year activity.

The process evaluation was based on responses from six of the 39 teacher trials, limiting how far the implementation findings can be generalized.

Despite those limitations, the primary randomized analysis found a statistically significant overall improvement for students assigned to Medly, with positive effects recorded independently across all three GCSE science subjects and no evidence of a significantly different treatment effect between Pupil Premium and non-Pupil Premium pupils.

WhatWorked Education concludes that the findings support continued use of Medly as a revision tool while providing a basis for further testing.

A larger replication study is planned for the next academic year, with the next phase intended to test whether the attainment effect holds across a bigger sample and provide further evidence on student outcomes and implementation in schools.