How are AI models tested for dangerous capabilities like bioweapon instructions?

Numbers match the citations in the article. A working link isn't proof that a page supports a claim; check the quoted passage and date.

1. 1

   Evaluating Frontier Models for Dangerous Capabilities

   arXiv (Cornell University) (Phuong et al.)Published Mar 20, 2024Checked Oct 4, 2026

   “To understand the risks posed by a new AI system, we must understand what it can and cannot do. Building on prior work, we introduce a programme of new "dangerous capability" evaluations and pilot them on Gemini 1.0 models. Our evaluations cover four areas: (1) persuasion and deception; (2) cyber-security; (3) self-proliferation; and (4) self-reasoning. We do not find evidence of strong dangerous capabilities in the models we evaluated, but we flag early warning signs. Our goal is to help advance a rigorous science of dangerous capability evaluation, in preparation for future models.”

2. 2

   From capability uplift to capability governance: an AI-biosecurity stack.

   Frontiers in microbiology (Luhachack et al.)Published Aug 14, 2026Checked Oct 4, 2026

   “"Capability uplift," or ΔAI, is a term used in the report to assess how AI-enabled biological tools can uniquely, and in some cases, specifically enable increases in or changes to biosecurity risks. The report proposes an "if-then" approach to monitor emerging capabilities through observable indicators such as new datasets as the leading indicator of capability, model performance, and the erosion of build/test barriers. Since the report's publication, agentic AI systems, virtual scientific teams, genome-scale foundation models, and self-driving laboratories have advanced from largely prospective concerns to early demonstrations. Multi-agent systems have been reported for biomedical hypothesis generation, design, and semi-autonomous discovery workflows, while self-driving laboratories now are considered as practical platforms for biotechnology. This perspective article extends these insights by employing a conceptual framework analysis and involves: (1) categorizing key AI capabilities across the DBTL cycle into a layered capability stack, and (2) illustrate how the if-then approach can be used to inform a dashboard based on observable indicators.”

3. 3

   Red-teaming as an imperative for strengthening synthetic nucleic acid screening.

   Frontiers in bioengineering and biotechnology (Khan & Thorat)Published Jun 24, 2026Checked Oct 4, 2026

   “Current oversight mechanisms rely on sequence-of-concern (SoC) screening, customer vetting, and voluntary compliance frameworks. However, these strategies have documented limitations, including static SoC databases, bypass pathways, and gaps that widen as SNAT capabilities become more distributed and adaptable. The literature contains multiple examples of how existing safeguards can miss dangerous constructs. This makes it imperative to identify shortcomings in extant screening tools and rethink SNAT regulatory strategies toward more proactive and dynamic approaches. Red-teaming, widely used in military planning, cybersecurity, and risk management, employs controlled adversarial testing to identify vulnerabilities before they can be exploited. There has been little effort to consolidate or assess the broader potential of red-teaming in the context of SNAT biosecurity. This article will examine current red-teaming approaches relevant to SNAT biosecurity, analyse how these methods have been applied in adjacent fields, and outline how similar principles could support more adaptive and dynamic screening frameworks.”

4. 4

   Dual-use capabilities of concern of biological AI models.

   PLoS computational biology (Pannu et al.)Published May 8, 2025Checked Oct 4, 2026

   “Of these dual-use capabilities, we argue that AI model evaluations should prioritize addressing those which enable high-consequence risks (i.e., large-scale harm to the public, such as transmissible disease outbreaks that could develop into pandemics), and that these risks should be evaluated prior to model deployment so as to allow potential biosafety and/or biosecurity measures. While biological research is on balance immensely beneficial, it is well recognized that some biological information or technologies could be intentionally or inadvertently misused to cause consequential harm to the public. AI-enabled life sciences research is no different. Scientists' historical experience with identifying and mitigating dual-use biological risks can thus help inform new approaches to evaluating biological AI models. Identifying which AI capabilities pose the greatest biosecurity and biosafety concerns is necessary in order to establish targeted AI safety evaluation methods, secure these tools against accident and misuse, and avoid impeding immense potential benefits.”

5. 5

   AI Sandbagging: Language Models can Strategically Underperform on Evaluations

   arXiv (Cornell University) (Weij et al.)Published Jun 11, 2024Checked Oct 4, 2026

   “These conflicting interests lead to the problem of sandbagging, which we define as strategic underperformance on an evaluation. In this paper we assess sandbagging capabilities in contemporary language models (LMs). We prompt frontier LMs, like GPT-4 and Claude 3 Opus, to selectively underperform on dangerous capability evaluations, while maintaining performance on general (harmless) capability evaluations. Moreover, we find that models can be fine-tuned, on a synthetic dataset, to hide specific capabilities unless given a password. This behaviour generalizes to high-quality, held-out benchmarks such as WMDP. In addition, we show that both frontier and smaller models can be prompted or password-locked to target specific scores on a capability evaluation. We have mediocre success in password-locking a model to mimic the answers a weaker model would give. Overall, our results suggest that capability evaluations are vulnerable to sandbagging. This vulnerability decreases the trustworthiness of evaluations, and thereby undermines important safety decisions regarding the development and deployment of advanced AI systems.”

6. 6

   FUSE: An Evaluating Framework for Dangerous Capabilities of LLMs

   arXiv (Cornell University) (Jin et al.)Published Sep 2, 2026Checked Oct 4, 2026

   “Instantiating the framework with a chemical-biological (CB) module, we evaluate 12 commercial LLMs from four families. Our first contribution is a horizontal comparison of dangerous capability across models and model families: the three dimensions expose sharply divergent profiles---models with comparable knowledge differ in refusal resilience, and strong defenders do not generate less harmful content when they do comply---while family-level patterns further separate Claude, DeepSeek, and GPT models. The second is a temporal analysis of capability evolution: tracking $K$, $D$, and $H$ against model release dates reveals that dangerous capability has not monotonically declined; newer models deepen knowledge while only partially improving defense, showing that scaling and alignment progress do not uniformly translate into safety. Reliability is established via cross-judge consistency (bootstrap $ρ> 0.79$, 4 of 5 judges) and pipeline orthogonality ($K$--$D$--$H$ inter-correlations $ρ\in [0.32, 0.52]$).”

7. 7

   Without safeguards, AI-Biology integration risks accelerating future pandemics.

   Frontiers in microbiology (Wang et al.)Published Jan 22, 2026Checked Oct 4, 2026

   “Artificial intelligence now shapes the design of biological matter. Protein language models (pLMs), trained on millions of natural sequences, can predict, generate, and optimize functional proteins with minimal human input. When embedded in experimental pipelines, these systems enable closed-loop biological design at unprecedented speed. The same convergence that accelerates vaccine and therapeutic discovery, however, also creates new dual-use risks. We first map recent progress in using pLMs for fitness optimization across proteins, then critically assess how these approaches have been applied to viral evolution and how they intersect with laboratory workflows, including active learning and automation. Building on this analysis, we outline a capability-oriented framework for integrated AI-biology systems, identify evaluation challenges specific to biological outputs, and propose research directions for training- and inference-time safeguards.”

8. 8

   Cyber-biological convergence: a systematic review and future outlook.

   Frontiers in bioengineering and biotechnology (Elgabry & Johnson)Published Sep 24, 2024Checked Oct 4, 2026

   “This includes cyber-bio opportunities and threats as engineered biology continues to integrate into cyberspace. We used a systematic search methodology to review the academic literature, and supplemented this with a review of opensource materials and "grey" literature that is not disseminated by academic publishers. A comprehensive search of articles published in or after 2017 until the 21st of October 2022 found 52 studies that focus on implications of engineered biology to cyberspace. The search was conducted using search engines that index over 60 databases-databases that specifically cover the information security, and biology literatures, as well as the wider set of academic disciplines. Across these 52 articles, we identified a total of 7 cyber opportunities including automated bio-foundries and 4 cyber threats such as Artificial Intelligence misuse and biological dataset targeting. We highlight the 4 main types of cyberbiosecurity solutions identified in the literature and we suggest a total of 9 policy recommendations that can be utilized by various entities, including governments, to ensure that cyberbiosecurity remains frontline in a growing bioeconomy.”

9. 9

   Quantifying detection rates for dangerous capabilities: a theoretical model of dangerous capability evaluations

   RePEc: Research Papers in Economics (Bova et al.)Published Dec 19, 2024Checked Oct 4, 2026

   “We first use the model to provide a novel introduction to dangerous capability testing and how this testing can directly inform policy. Decision makers in AI labs and government often set policy that is sensitive to the estimated danger of AI systems, and may wish to set policies that condition on the crossing of a set threshold for danger. The model helps us to reason about these policy choices. We then run simulations to illustrate how we might fail to test for dangerous capabilities. To summarise, failures in dangerous capability testing may manifest in two ways: higher bias in our estimates of AI danger, or larger lags in threshold monitoring. We highlight two drivers of these failure modes: uncertainty around dynamics in AI capabilities and competition between frontier AI labs. Effective AI policy demands that we address these failure modes and their drivers. Even if the optimal targeting of resources is challenging, we show how delays in testing can harm AI policy. We offer preliminary recommendations for building an effective testing ecosystem for dangerous capabilities and advise on a research agenda.”

10. 10

    Expanding External Access To Frontier AI Models For Dangerous Capability Evaluations

    arXiv (Cornell University) (Charnock et al.)Published Jan 17, 2026Checked Oct 4, 2026

    “Furthermore, there is no common framework for describing different types and levels of evaluator access. To address this gap, we propose a taxonomy of access methods for dangerous capability evaluations. We disentangle three aspects of access: model access, model information, and evaluation timeframe. For each aspect, we review benefits and risks, including how expanding access can reduce false negatives and improve stakeholder trust, but can also increase security and capacity challenges. We argue that these limitations can likely be mitigated through technical means and safeguards used in other industries. Based on the taxonomy, we propose three descriptive access levels: AL1 (black-box model access and minimal information), AL2 (grey-box model access and substantial information), and AL3 (white-box model access and comprehensive information), to support clearer communication between evaluators, frontier AI companies, and policymakers. We believe these levels correspond to the different standards for appropriate access defined in the EU Code of Practice, though these standards may change over time.”