Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM Novice Uplift on Dual-Use, In Silico Biology Tasks

Chen Bo Calvin Zhang, Christina Q. Knight, Nicholas Kruus, Jason Hausenloy, Pedro Medeiros, Nathaniel Li, Aiden Kim, Yury Orlovskiy, Coleman Breen, Bryce Cai, Jasper Götting, Andrew Bo Liu, Samira Nedungadi, Paula Rodriguez, Yannis Yiming He, Mohamed Shaaban, Zifan Wang, Seth Donoughe, Julian Michael

TL;DR

Overall, LLMs substantially uplift novices on biological tasks previously reserved for trained practitioners, underscoring the need for sustained, interactive uplift evaluations alongside traditional benchmarks.

Abstract

Large language models (LLMs) perform increasingly well on biology benchmarks, but it remains unclear whether they uplift novice users -- i.e., enable humans to perform better than with internet-only resources. This uncertainty is central to understanding both scientific acceleration and dual-use risk. We conducted a multi-model, multi-benchmark human uplift study comparing novices with LLM access versus internet-only access across eight biosecurity-relevant task sets. Participants worked on complex problems with ample time (up to 13 hours for the most involved tasks). We found that LLM access provided substantial uplift: novices with LLMs were 4.16 times more accurate than controls (95% CI [2.63, 6.87]). On four benchmarks with available expert baselines (internet-only), novices with LLMs outperformed experts on three of them. Perhaps surprisingly, standalone LLMs often exceeded LLM-assisted novices, indicating that users were not eliciting the strongest available contributions from the LLMs. Most participants (89.6%) reported little difficulty obtaining dual-use-relevant information despite safeguards. Overall, LLMs substantially uplift novices on biological tasks previously reserved for trained practitioners, underscoring the need for sustained, interactive uplift evaluations alongside traditional benchmarks.

LLM Novice Uplift on Dual-Use, In Silico Biology Tasks

TL;DR

Overall, LLMs substantially uplift novices on biological tasks previously reserved for trained practitioners, underscoring the need for sustained, interactive uplift evaluations alongside traditional benchmarks.

Abstract

Large language models (LLMs) perform increasingly well on biology benchmarks, but it remains unclear whether they uplift novice users -- i.e., enable humans to perform better than with internet-only resources. This uncertainty is central to understanding both scientific acceleration and dual-use risk. We conducted a multi-model, multi-benchmark human uplift study comparing novices with LLM access versus internet-only access across eight biosecurity-relevant task sets. Participants worked on complex problems with ample time (up to 13 hours for the most involved tasks). We found that LLM access provided substantial uplift: novices with LLMs were 4.16 times more accurate than controls (95% CI [2.63, 6.87]). On four benchmarks with available expert baselines (internet-only), novices with LLMs outperformed experts on three of them. Perhaps surprisingly, standalone LLMs often exceeded LLM-assisted novices, indicating that users were not eliciting the strongest available contributions from the LLMs. Most participants (89.6%) reported little difficulty obtaining dual-use-relevant information despite safeguards. Overall, LLMs substantially uplift novices on biological tasks previously reserved for trained practitioners, underscoring the need for sustained, interactive uplift evaluations alongside traditional benchmarks.
Paper Structure (82 sections, 1 equation, 33 figures, 6 tables)

This paper contains 82 sections, 1 equation, 33 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Our Contribution
  3. Extended Interactions
  4. Multiple Models
  5. Diverse Questions & Actors
  6. Qualitative Analysis
  7. Related Work
  8. Dual-Use Biological Capabilities.
  9. Biological Capability Uplift.
  10. Benchmarking Biorisk.
  11. Challenges of Current Benchmarks.
  12. Qualitative Analysis of Human-LLM Interaction.
  13. Method
  14. Participants
  15. Experimental Design
  16. ...and 67 more sections

Figures (33)

  • Figure 1: Experimental design and resulting performance scores.
  • Figure 2: Task accuracy across benchmarks. LLM assistance substantially elevates novice performance, often bringing Treatment accuracy close to or beyond that of LLM-only systems and human experts.
  • Figure 3: Confidence calibration. Both conditions exhibit overconfidence (curves below the diagonal), but Treatment participants are better calibrated than Controls, particularly at moderate-to-high confidence.
  • Figure 4: Usage of the Deep Research Feature. Among tasks in the treatment group, 37.5% utilized the 'Deep Research' feature, while the remaining 62.5% were completed without it.
  • Figure 5: Model Usage Distribution. The chart illustrates the percentage of total user messages handled by each available large language model. Gemini 2.5 Pro was the most frequently used model, accounting for 33.5% of all messages.
  • ...and 28 more figures