Safety failure
Realized harm
A Waymo detected untraversable floodwater, slowed but did not stop, drove in and was swept into a San Antonio waterway
AIR-2026-0093 AIR record · 2026-04-waymo-san-antonio-flood-sweep
A real party was affected. This incident caused actual consequences, not a demonstrated capability.
From the supporting source
"On higher speed roadways, the Waymo AV may slow but not stop in response to detecting a potentially untraversable flooded lane," NHTSA recall documents read.abc7news.com
The record
Disclosed
2026-04-21
Occurred
2026-04-20
Vendor
Waymo (Alphabet)
Product
Waymo 5th and 6th Generation Automated Driving Systems
Risk domain
Agent behaviour › Unsafe actuation
System type
Classical autonomy (no generative model)
Who acted
Agent acted — the vehicle drove into the floodwater it had detected
Attack surface
Autonomous operations
Autonomy level
Fully autonomous
Initial vector
None autonomous action
Impact type
Property damage
Confidence
High
Submitted by
EAI Internal
Crosswalk
- OWASP Agentic Top 10 (2026)
- ASI01 – Agent Goal Hijack
- MITRE ATLAS
- AML.T0048 – External Harms
- NIST AI RMF
- MEASURE 2.6; MANAGE 1.3
- EU AI Act
- Annex III §1(c)
- ISO/IEC
- TR 24028; 24029-1; 23894
- AIUC-1
- C. Safety
- Scope
- agent misuse
- Mitigation
- Require confirmation for destructive actions; Implement dry-run modes; Use layered safety controls.
- Taxonomy
- Enkrypt AI agent risk taxonomy · Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery
Blast radius
one vehicle lost near Salado Creek; 3,791 automated driving systems recalled; no occupants
Capability abused
Defense & remediation
Recorded defense
flooded-lane detection
Guardrail kinds
Unknown
Guardrail outcome
Silent
Fix shipped
San Antonio operations suspended; voluntary NHTSA recall of up to 3,791 vehicles
Taxonomy & references
Curator notes
The recall language is the cleanest statement of a recurring agent-design fault in this corpus: the system correctly perceived a hazard and its policy converted that perception into caution rather than refusal. Detection without a hard stop is not a safety control. Same gap as the Zoox door collision and the Jiushi drag, here written down by the regulator. AI Incident Database incident 1544.
Supporting sources
Cite as
Enkrypt AI. "A Waymo detected untraversable floodwater, slowed but did not stop, drove in and was swept into a San Antonio waterway". AIR, AIR-2026-0093. https://agentincident.vercel.app/i/AIR-2026-0093/ (accessed 2026-10-19).