Skip to content

BioBasicsBench AI Benchmark: OpenAI Tests 100 Real-World Lab Science Tasks

OpenAI's BioBasicsBench evaluates AI models on 100 real-world laboratory analysis tasks. Its results put GPT-6 Astra ahead of GPT-5.6 Sol while revealing important gaps in scientific reliability and independent verification.

BioBasicsBench AI Benchmark: OpenAI Tests 100 Real-World Lab Science Tasks

On this page

OpenAI's published results show that GPT-6 Astra performed better than GPT-5.6 Sol while producing considerably fewer output tokens. The company attributes part of the improvement to stronger multimodal capabilities, although the benchmark does not isolate the cause. The GPT-6 Astra model documentation describes its reasoning capabilities and supported image input, providing useful context for interpreting the result.

Why 100 ordinary laboratory tasks make a different kind of AI test

BioBasicsBench focuses on five areas of laboratory science: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. These areas cover work such as examining protein gels, interpreting microscopy images, measuring experimental results and evaluating molecular biology procedures. The benchmark uses raw evidence, including gel images, colony plates, microscopy and instrument readouts. That distinction matters because scientific data rarely arrive with every uncertainty already resolved. A researcher may need to identify an unexpected result, check whether the experiment included suitable controls and decide whether the available evidence is strong enough to support the original hypothesis.

OpenAI developed the benchmark with Ligo, a biotechnology company focused on enzyme design. According to the BioBasicsBench announcement, its tasks assess three closely connected abilities: interpreting experimental data, identifying errors and recognising when the evidence cannot justify a conclusion. The third ability is particularly important. An AI assistant that can say a result is inconclusive may be more useful to a scientist than one that always produces a confident explanation, especially when an incorrect interpretation could send an experiment in the wrong direction.

Why 100 ordinary laboratory tasks make a different kind of AI test

The important detail is how the benchmark treats uncertainty

The most interesting part of BioBasicsBench is not simply whether an AI model can identify objects in laboratory images. It is whether the model can distinguish a convincing-looking observation from a scientifically reliable result. Experimental controls are checks that help researchers determine whether an observed effect comes from the factor being investigated rather than an unrelated influence. If a control is missing, incorrectly prepared or inconsistent with the expected result, an otherwise plausible interpretation may be wrong. OpenAI says its evaluation includes situations in which agents must recognise problems with experimental controls and avoid conclusions that the available evidence cannot support.

This makes the benchmark different from a conventional image-recognition test. Identifying a band on a gel image is one task; understanding whether that band supports a particular biological interpretation is another. Similarly, recognising colonies on a plate does not establish that the correct experimental condition produced them. A useful scientific assistant must connect visual observations with experimental context, not treat every image as an isolated classification problem. BioBasicsBench therefore addresses a practical weakness in AI-assisted research: the distance between recognising laboratory data and interpreting what those data actually establish.

GPT-6 Astra scores 66.7%, but the comparison needs context

OpenAI evaluated seven models across 37 combinations of models and reasoning-effort settings. Reasoning effort refers to how much computational work a model is allowed to spend considering a task before producing its answer. Each setting was evaluated on all 100 tasks, with five attempts per task, producing 18,500 results. OpenAI reports mean reward, which represents the average proportion of available scoring credit earned across the benchmark. It is not a percentage of experiments successfully completed, nor does it mean that a model independently conducted 66.7% of laboratory work correctly. OpenAI's model specifications also identify xhigh as one of the available reasoning-effort settings.

At the xhigh reasoning setting, the highest common setting reported for all seven models, GPT-6 Astra achieved 66.7% mean reward. GPT-6.1 Sol followed at 62.0%, while GPT-5.6 Sol recorded 43.8%. The difference between Astra and the older GPT-5.6 Sol is 22.9 percentage points. That is a substantial improvement within this evaluation, although it does not establish that Astra is 22.9% more accurate in every scientific application. The comparison is limited to the particular tasks, scoring rules and model configurations used by OpenAI.

AI modelMean reward95% confidence intervalOutput tokens per task
GPT-6 Astra66.7%60.3–73.1%4,704
GPT-6.1 Sol62.0%55.6–68.4%5,238
GPT-5.6 Sol43.8%37.6–50.0%9,099
GPT-6 Luna38.3%32.3–44.5%13,120
GPT-5.535.4%29.7–41.0%11,135
GPT-5.433.2%28.3–38.4%13,717
GPT-5.231.2%26.2–36.3%27,620

The uncertainty ranges are worth examining alongside the headline scores. OpenAI reports 95% confidence intervals, which estimate the uncertainty around the measured average using resampling of complete tasks while keeping each task's five attempts together. Astra's interval runs from 60.3% to 73.1%, compared with 37.6% to 50.0% for GPT-5.6 Sol. These intervals do not overlap, supporting a clear separation in this particular evaluation. However, Astra's interval overlaps substantially with GPT-6.1 Sol's 55.6% to 68.4% range, so the 4.7-point difference between those two models should not be treated as decisive evidence that Astra is better in every comparable situation.

GPT-6 Astra scores 66.7%, but the comparison needs context

The token figures reveal another part of the result

There is a second finding in the evaluation that deserves attention: the strongest result did not require the longest answers. GPT-6 Astra produced an average of 4,704 output tokens per task at xhigh, compared with 9,099 for GPT-5.6 Sol and 27,620 for GPT-5.2. Output tokens are the units of generated text, so this measurement provides a rough indication of how much the models wrote, not a direct measure of their total computational cost. Astra earned a higher score than GPT-5.6 Sol while generating roughly half as many output tokens. Compared with GPT-5.2, it used about one-sixth as many.

OpenAI suggests that stronger native multimodal capabilities may explain part of the improvement. Multimodal AI can work with different forms of information, such as text and images, rather than depending exclusively on written descriptions. That matters in a laboratory setting because evidence often arrives as an image or instrument output that needs to be interpreted directly. The result raises an interesting possibility: improving how a model understands scientific images and their surrounding context may be more valuable than simply encouraging it to produce longer reasoning responses. However, the token comparison alone does not prove that image understanding caused the score improvement. Establishing that would require controlled experiments that isolate the contribution of individual capabilities.

How this evaluation differs from earlier model comparisons

Many AI benchmarks concentrate on questions with established answers, coding problems or scientific information presented in a clean format. BioBasicsBench instead focuses on the evidence-handling stage that sits between conducting an experiment and writing up its findings. Its tasks are designed around the imperfections that appear during everyday laboratory work, including ambiguous images, unexpected observations and problematic controls. That makes the benchmark relevant to a different question: can an AI assistant help a scientist reason responsibly about the material already collected, rather than merely recall scientific knowledge?

The evaluation also illustrates why model comparisons need more than a single leaderboard number. A model can score well because it recognises common visual patterns, yet struggle when the experimental conditions change. Another model may produce a plausible explanation but fail to identify that a control does not support the conclusion. Testing repeated attempts across the same tasks provides a broader view than selecting a few impressive examples, but the quality of the benchmark's scoring criteria remains central. The results should therefore be read as evidence of progress on a defined set of laboratory interpretation problems, not as a general ranking of scientific intelligence.

What BioBasicsBench does not establish yet

The reported results do not show that GPT-6 Astra can independently conduct laboratory experiments, operate scientific instruments or design a successful research programme. The benchmark evaluates the interpretation of supplied experimental evidence, not the complete process of biological discovery. Nor does a 66.7% mean reward establish that the model is ready to make unsupervised scientific decisions. A human scientist still needs to check the underlying evidence, validate important interpretations and decide whether a proposed conclusion is scientifically justified.

There is also a reproducibility question. The November 6 announcement presents OpenAI's own evaluation results, but does not provide a public task dataset, complete scoring rubrics or an independent replication that would allow outside researchers to reproduce the reported scores. The published comparison covers seven OpenAI models rather than a broad selection of competing systems. Without an independently measured human baseline, it is also impossible to determine from these figures alone how close the models are to experienced laboratory scientists. These limitations do not make the benchmark unhelpful; they define what its results can legitimately support.

Independent evaluation is the next important milestone

The next useful step would be making enough of BioBasicsBench available for independent researchers to inspect its task design, scoring criteria and model outputs. A reproducible evaluation could establish whether the reported performance differences remain consistent outside the original development environment. Including other AI providers and experienced human researchers would also help clarify whether the benchmark measures a meaningful level of laboratory competence or mainly captures differences between model families. Until that evidence exists, laboratories should treat the scores as a promising research result rather than a deployment recommendation.

For scientists considering AI-assisted analysis, the practical question is not whether a model can produce an impressive explanation of an experiment. It is whether it can identify a weak control, explain uncertainty and recognise when the available data are insufficient. BioBasicsBench puts those questions closer to the centre of AI evaluation. Its next real test will be whether independent laboratories can reproduce the findings and establish where an AI assistant genuinely saves scientific time without weakening the standards used to judge experimental evidence.

M. Rizwan Mirza profile photo

Written by

M Rizwan Mirza

I’m M Rizwan Mirza, a Full Stack Developer with over 12 years of experience in web development and software solutions. I work with modern web technologies and enjoy building practical, reliable, and user-friendly digital solutions. I’m also part of TechWare House, where I work on web development projects and technology solutions. One of my favorite websites is TheQuranic.com. Through WizTechnoz, I share my knowledge, experience, tutorials, and useful insights about technology.

86 posts published

All posts by this author

Copy this post

0 Comments

No comments yet. Be the first to share your thoughts.

Join the conversation

Log in or create a free account to leave a comment. You can edit or delete your own comments any time.