Developed with Ligo Biosciences, BioBasicsBench uses 100 tasks from real experiments to test how AI agents interpret data, check controls and handle uncertainty
BioBasicsBench evaluates AI agents’ interpretation of laboratory data using 100 tasks drawn from real experiments. Conceptual illustration.
OpenAI’s GPT-6 Astra earned 66.7% of the available credit on a new laboratory analysis benchmark, compared with 43.8% for GPT-5.6 Sol, but still made counting errors and selected the wrong experimental samples.
The results come from BioBasicsBench, developed by OpenAI with enzyme design company Ligo Biosciences. It assesses how AI agents interpret evidence from wet labs, where scientists conduct experiments with biological samples.
Both figures were recorded at the “xhigh” reasoning setting, which allows models more reasoning per task. Scores reflect the average credit earned against expert-written grading criteria, including partial credit for elements of an answer that are correct.
The evaluation shows improvements in interpreting images and recognizing experimental problems. It also documents cases where Astra noticed relevant evidence but failed to carry it through to its final answer.
OpenAI cautions: “Higher benchmark scores do not yet mean models can make reliable judgments in the lab.”
Spotting problems that earlier models missed
Several examples show Astra recognizing limits in experimental evidence that GPT-5.6 Sol overlooked.
In one task, GPT-5.6 Sol classified cells as dead based on the brightness of a dye signal. However, the detector could also pick up light from a second dye, making brightness alone insufficient to support that conclusion. Astra identified the ambiguity and called for better controls to separate the signals.
Controls provide comparisons that help scientists check how to interpret an experiment. Reading them correctly is part of the benchmark, including noticing when a supposed control cannot be relied on.
In a DNA-cloning task, GPT-5.6 Sol based its conclusion on the absence of colonies from a negative control plate. Astra spotted a detail in the photograph: the plate was labeled with a different antibiotic from the one used in the experiment. It rejected the plate as a reliable control.
Another task involved a sequence of microscope images in which cells appeared to disappear and reappear. GPT-5.6 Sol looked for a biological explanation. Astra correctly recognized that the images showed a depth scan, with different parts of the sample coming into focus as the microscope moved through it.
OpenAI reports broader improvements in Astra’s handling of photographs, videos and raw outputs from laboratory instruments, although its examples also show persistent failures.
Wrong samples, ignored evidence and counting mistakes
One failure involved choosing samples from a DNA-cloning experiment.
Astra correctly distinguished two groups of bands in the experimental results, but attached the wrong meaning to them. It assumed that a larger DNA product indicated successful cloning, overriding the interpretation supported by the controls.
The result was a selection of the wrong colonies for follow-up work. OpenAI says that choice would have led to purification of the incorrect plasmid.
In a microscopy task, Astra inspected compartments inside cells called vacuoles. Some appeared darker than their surroundings and others appeared lighter, with changes in focus affecting the contrast.
The model noticed that contrast varied with focus. Its final answer nevertheless described the compartments as generally darker, leaving out observations that contradicted its preferred interpretation.
A separate task exposed problems with a routine measurement: counting colonies on a plate.
Astra failed to separate closely spaced colonies, missed some visible colonies and marked empty areas as colonies. On that image, the combined effect was an overcount of approximately 12% relative to the benchmark’s reference count.
These examples identify different points of failure: interpreting what an image shows, applying experimental context and ensuring that the final conclusion reflects all the evidence examined.
100 tasks built from scientists’ own experiments
BioBasicsBench covers five areas of routine laboratory work, spanning protein methods, imaging and cell measurement, laboratory measurements and assays, nucleic acid methods, and molecular cloning.
The inputs include microscope images, gel and blot photographs, sequencing files, spreadsheets, handwritten laboratory notes, instrument displays and videos of bench work. Some tasks combine several types of evidence.
OpenAI argues that benchmarks using charts and images selected for scientific papers can miss the ambiguity of everyday experiments, including failed procedures, unclear images and conflicting observations.
Scientists created BioBasicsBench tasks from their own experimental data and wrote grading criteria for each answer. Each task underwent review by another Ligo scientist, an external PhD-level scientist working in a wet lab, and a final internal reviewer.
The evaluation covered seven OpenAI models across 37 combinations of model and reasoning settings. Each setting involved five attempts at every task, producing 18,500 results.
At “xhigh,” GPT-6.1 Sol earned 62.0% of the available credit, while GPT-6 Luna received 38.3%. The benchmark grades individual elements of an answer, including its conclusion, numerical results, supporting reasoning and recognition that the evidence is insufficient.
OpenAI presents the benchmark as a way to track improvements in scientific assistance and their implications for biosafety and biosecurity. It describes the intended benefit cautiously: “Progress on these tasks could help scientists spend less time troubleshooting and more time deciding which experiments to pursue next.”
ETIH Innovation Awards 2027 open for entries
The ETIH Innovation Awards 2027 are now open for entries, with 26 categories recognising companies, institutions, projects and individuals making a measurable impact across education technology. The global awards span AI in education, digital learning, higher education, skills and workforce, inclusion, education operations, evidence and impact, with entries managed through ETIH’s new online awards platform.
Entries are open worldwide, with the early-bird deadline on Friday 18 December 2026 and the final entry deadline on Friday 26 February 2027 at 23:59 UK time.
Explore the 2027 categories and judging criteria, read the How to Enter guide or start an entry through the ETIH Innovation Awards 2027 platform.
