GPT-4 Gives Biologists a Mild Edge in Threat Creation Study

OpenAI has released early results from a study measuring whether GPT-4 meaningfully increases the ability to create biological threats, and the findings are a mixed bag leaning toward a reassuring shrug. The company ran 100 participants—half PhD-level experts with wet lab experience, half students with at least one biology course—through a gauntlet of tasks covering the end-to-end biothreat creation process. One group got only internet access; the other got the internet plus GPT-4.

The raw numbers do show an uplift. On a 10-point accuracy scale, experts with the model scored 0.88 points higher on average than their internet-only peers. Students saw a bump of just 0.25 points—barely a nudge. Completeness scores followed a similar pattern, with expert gains of 0.82 and students at 0.41. But here’s the headline caveat that should temper any panic: none of these results reached statistical significance. The effect sizes simply weren’t large enough to draw a firm conclusion, which OpenAI frames as a starting point rather than a verdict.

What’s genuinely new here is the methodology itself. This is the largest human evaluation of AI’s impact on biorisk information to date, and OpenAI is openly treating it as a tripwire test under its Preparedness Framework. The company notes that measuring statistical significance alone may be a poor yardstick for model risk, calling for new research into what performance thresholds actually signal a meaningful increase in danger. They’re also careful to stress that information access is not the same as building a weapon—this study didn’t test physical construction.

The broader context is a policy conversation that has been heavy on hypotheticals and light on data. The White House, researchers, and think tanks have all warned about AI models serving as bioterrorism tutors, but empirical evaluations have been sparse. OpenAI’s willingness to share methods and invite community feedback signals a rare openness, though the modest results leave the core question unresolved. The model might make experts slightly faster or more thorough at research, but for now, the internet remains the more dangerous adversary.