How is AI tested for dangerous capabilities like bioweapon instructions?

Numbers match the citations in the article. A working link isn't proof that a page supports a claim; check the quoted passage and date.

1. 1

   Evaluating Frontier Models for Dangerous Capabilities

   arXiv (Cornell University) (Phuong et al.)Published Mar 20, 2024Checked Oct 5, 2026

   “To understand the risks posed by a new AI system, we must understand what it can and cannot do. Building on prior work, we introduce a programme of new "dangerous capability" evaluations and pilot them on Gemini 1.0 models. Our evaluations cover four areas: (1) persuasion and deception; (2) cyber-security; (3) self-proliferation; and (4) self-reasoning. We do not find evidence of strong dangerous capabilities in the models we evaluated, but we flag early warning signs. Our goal is to help advance a rigorous science of dangerous capability evaluation, in preparation for future models.”

2. 2

   A Blind Spot in Alignment: Quantifying Biosecurity Risks in Large Language Models

   arXiv (Cornell University) (Quan et al.)Published Aug 3, 2026Checked Oct 5, 2026

   “Current safety evaluations, however, operate in natural language and cannot determine whether a model-generated amino acid sequence is biological gibberish or a computational risk signal. To address this evaluation blind spot, we introduce SPIKE-Bench, coupling 631 curated toxin-design prompts across seven functional categories with the SPIKE funnel, a three-stage protocol that filters output through compliance, biological plausibility, and predicted toxicity, producing stage-level diagnostics and an aggregate function-aware metric: the Functional Harmfulness Rate (FHR). An audit of 32 LLMs reveals that most models freely comply with toxin-design requests; FHR is driven primarily by biological generation capability rather than safety alignment, reaching 50.7%; and Refusal Rate fails to predict functional risk. As a first step toward mitigation, we provide BioSafe-Guard, a domain-specialized classifier that substantially reduces predicted functional risk while preserving benign utility. We release SPIKE-Bench and BioSafe-Guard at https://github.com/PKU-Alignment/SPIKE-Bench to support more rigorous biosecurity evaluation of LLMs.”

3. 3

   BioSecBench-Refusal: A paired metric for performance and alignment in agentic biosecurity risk assessment

   arXiv (Cornell University) (Wintermute et al.)Published Jul 6, 2026Checked Oct 5, 2026

   “As AI agents are incorporated into life science workflows, the capabilities that speed discovery might also enable misuse. We present BioSecBench-Refusal, a benchmark for risk identification and refusal behavior for biological research tasks. The benchmark pairs 61 Routine tasks, legitimate analyses adapted from the published literature, with 46 Red-Team tasks, fictional scenarios that resemble real research but conceal a biosecurity hazard. Across 16 model-harness configurations, refusal rates ranged from 7 percent to 74 percent on Routine tasks and 1 percent to 62 percent on Red-Team tasks, with many configurations refusing legitimate Routine work at comparable or higher rates than concealed hazards. Refusals were most often triggered by provider API filters applied prior to agentic reasoning. However, models given room to reason showed the potential to identify more real threats. We release BioSecBench-Refusal as a tool for model developers to calibrate capability and caution for agentic biotech research and development.”

4. 4

   Biosecurity Risk Assessment for the Use of Artificial Intelligence in Synthetic Biology

   Applied Biosafety (Haro)Published Mar 27, 2024Checked Oct 5, 2026

   “The tools and methodology provided offer a structured approach to risk assessment, enabling risk management professionals to comprehensively analyze the biosecurity implications of AI applications in synthetic biology. They facilitate the identification of potential risks and the development of effective mitigation strategies. An example of a risk assessment performed on the large language model "ChatGPT 4.0" is provided here. AI's role in synthetic biology is rapidly expanding; thus, establishing proactive and secure practices is crucial. The biosecurity risk assessment tools and methodology presented here are the first provided in the literature and will be instrumental steps toward the responsible integration of AI in synthetic biology.”

5. 5

   An Early Warning of Emerging Biosecurity Risks in Frontier LLMs

   arXiv (Cornell University) (He et al.)Published Jul 20, 2026Checked Oct 5, 2026

   “Our evaluation reveals a concerning gap between text-level safeguards and the risks posed by capable scientific models: (i) Intern-BioBreaker outperforms baseline attack models and reveals widespread bio-risk jailbreak vulnerabilities across both open-weight and proprietary frontier LLMs, with several targets reaching near-saturated or 100% task-level attack success rate (ASR); (ii) in sequence-level case studies, GPT-5.5 can be induced to generate modified viral candidate sequences with pathogenic potential; the corresponding translated proteins may exhibit even stronger receptor-binding affinity and thus enhanced infection potential; and (iii) end-to-end verification shows that selected model-generated biological designs are not merely textual artifacts, but can be physically realized under controlled experimental settings. These findings underscore the need for stronger biological red-teaming, nucleic acid synthesis screening, and safety mechanisms that keep pace with model capabilities.”

6. 6

   Dual-use capabilities of concern of biological AI models.

   PLoS computational biology (Pannu et al.)Published May 8, 2025Checked Oct 5, 2026

   “Of these dual-use capabilities, we argue that AI model evaluations should prioritize addressing those which enable high-consequence risks (i.e., large-scale harm to the public, such as transmissible disease outbreaks that could develop into pandemics), and that these risks should be evaluated prior to model deployment so as to allow potential biosafety and/or biosecurity measures. While biological research is on balance immensely beneficial, it is well recognized that some biological information or technologies could be intentionally or inadvertently misused to cause consequential harm to the public. AI-enabled life sciences research is no different. Scientists' historical experience with identifying and mitigating dual-use biological risks can thus help inform new approaches to evaluating biological AI models. Identifying which AI capabilities pose the greatest biosecurity and biosafety concerns is necessary in order to establish targeted AI safety evaluation methods, secure these tools against accident and misuse, and avoid impeding immense potential benefits.”