Safety failure
Realized harm
A US Navy unmanned vessel under tow accelerated on an inadvertent command and capsized its support boat, throwing the captain into the water
AIR-2025-0051 AIR record · 2025-06-navy-usv-inadvertent-command-capsize
A real party was affected. This incident caused actual consequences, not a demonstrated capability.
From the supporting source
The Navy stopped a maritime drone test early and urgently requested support from the Coast Guard and local harbor patrol agents to help rescue a participating tugboat captain from waters off the California coast last week, multiple sources told DefenseScoop.defensescoop.com
The record
Disclosed
2025-07-01
Occurred
2025-06-23
Vendor
US Navy (operator); autonomy software vendor unnamed
Product
US Navy small unmanned surface vessel
Risk domain
Agent behaviour › Unsafe actuation
Also touches
Uncontrolled resource consumption
System type
Classical autonomy (no generative model)
Who acted
Agent acted — the unmanned vessel accelerated and capsized the support boat
Attack surface
Autonomous operations
Autonomy level
Semi-autonomous
Initial vector
Erroneous command injection
Impact type
Physical safety, Service disruption
Confidence
High
Submitted by
EAI Internal
Crosswalk
- OWASP Agentic Top 10 (2026)
- ASI01 – Agent Goal Hijack
- MITRE ATLAS
- AML.T0048 – External Harms
- NIST AI RMF
- MEASURE 2.6; MANAGE 1.3
- EU AI Act
- Annex III §1(c)
- ISO/IEC
- TR 24028; 24029-1; 23894
- AIUC-1
- C. Safety
- Scope
- agent misuse
- Mitigation
- Require confirmation for destructive actions; Implement dry-run modes; Use layered safety controls.
- Taxonomy
- Enkrypt AI agent risk taxonomy · Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery
Blast radius
one support boat capsized and its captain thrown into the water at Channel Islands Harbor Marina; Coast Guard and harbour patrol responded
Capability abused
Defense & remediation
Recorded defense
command authorisation between onboard systems and external autonomy software
Guardrail kinds
Unknown
Guardrail outcome
Silent
Fix shipped
test halted; incident attributed to software failures and human error including communication breakdowns between systems
Taxonomy & references
Curator notes
The one record in the corpus where the failure is squarely at the command interface rather than in perception or planning: an autonomy stack and an onboard control system disagreed about state, and the vessel accepted and executed a throttle command while under tow. No interlock tied 'I am being towed' to 'refuse propulsion commands'. Adjacent to prompt injection in structure — an unvalidated instruction from a trusted channel producing an unsafe physical action — with no adversary involved. AI Incident Database incident 1561.
Supporting sources
Cite as
Enkrypt AI. "A US Navy unmanned vessel under tow accelerated on an inadvertent command and capsized its support boat, throwing the captain into the water". AIR, AIR-2025-0051. https://agentincident.vercel.app/i/AIR-2025-0051/ (accessed 2026-10-19).