Safety failure
Realized harm
Waymo robotaxi struck a cyclist occluded by a truck at a San Francisco intersection
AIR-2024-0006 AIR record · 2024-02-waymo-potrero-hill-cyclist-collision
A real party was affected. This incident caused actual consequences, not a demonstrated capability.
From the supporting source
Waymo says the cyclist was blocked by a truck that had entered the intersection of 17th St. and Mississippi St., near the Potrero Hill and Mission Bay neighborhoods.abc7news.com
The record
Disclosed
2024-02-06
Occurred
2024-02-06
Vendor
Waymo (Alphabet)
Product
Waymo Driver
Risk domain
Agent behaviour › Unsafe actuation
System type
Classical autonomy (no generative model)
Who acted
Agent acted — the robotaxi struck the cyclist
Attack surface
Autonomous operations
Autonomy level
Fully autonomous
Initial vector
None autonomous action
Impact type
Physical safety
Confidence
High
Submitted by
EAI Internal
Crosswalk
- OWASP Agentic Top 10 (2026)
- ASI01 – Agent Goal Hijack
- MITRE ATLAS
- AML.T0048 – External Harms
- NIST AI RMF
- MEASURE 2.6; MANAGE 1.3
- EU AI Act
- Annex III §1(c)
- ISO/IEC
- TR 24028; 24029-1; 23894
- AIUC-1
- C. Safety
- Scope
- agent misuse
- Mitigation
- Require confirmation for destructive actions; Implement dry-run modes; Use layered safety controls.
- Taxonomy
- Enkrypt AI agent risk taxonomy · Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery
Blast radius
one cyclist, minor injuries
Capability abused
Defense & remediation
Recorded defense
perception and braking
Guardrail kinds
Physical safety system
Guardrail outcome
Silent
Fix shipped
none disclosed
Taxonomy & references
Curator notes
Occlusion case: the agent proceeded on an incomplete world model rather than yielding to what it could not see. Waymo published a study the following day claiming a 90% lower crash rate than human drivers, which is the tension this corpus keeps visible — aggregate safety gains and individual realized harms are both true. AI Incident Database incident 647.
Supporting sources
Cite as
Enkrypt AI. "Waymo robotaxi struck a cyclist occluded by a truck at a San Francisco intersection". AIR, AIR-2024-0006. https://agentincident.vercel.app/i/AIR-2024-0006/ (accessed 2026-10-19).