Safety failure
Realized harm
NHTSA opened a defect probe into ~600 Cruise driverless cars for failing to exercise caution around pedestrians and crosswalks
AIR-2023-0024 AIR record · 2023-10-cruise-nhtsa-pedestrian-probe
A real party was affected. This incident caused actual consequences, not a demonstrated capability.
From the supporting source
The National Highway Traffic Safety Administration said in a Tuesday filing that it had opened a safety-defect probe into nearly 600 driverless cars operated by Cruise, adding that they might not be exercising appropriate caution in crosswalks and roadways.wsj.com
The record
Disclosed
2023-10-17
Occurred
2023-10-17
Vendor
Cruise (General Motors)
Product
Cruise autonomous vehicle fleet
Risk domain
Agent behaviour › Unsafe actuation
System type
Classical autonomy (no generative model)
Who acted
Agent acted — ~600 vehicles drove without due caution around pedestrians
Attack surface
Autonomous operations
Autonomy level
Fully autonomous
Initial vector
None autonomous action
Impact type
Physical safety
Confidence
High
Submitted by
EAI Internal
Crosswalk
- OWASP Agentic Top 10 (2026)
- ASI01 – Agent Goal Hijack
- MITRE ATLAS
- AML.T0048 – External Harms
- NIST AI RMF
- MEASURE 2.6; MANAGE 1.3
- EU AI Act
- Annex III §1(c)
- ISO/IEC
- TR 24028; 24029-1; 23894
- AIUC-1
- C. Safety
- Scope
- agent misuse
- Mitigation
- Require confirmation for destructive actions; Implement dry-run modes; Use layered safety controls.
- Taxonomy
- Enkrypt AI agent risk taxonomy · Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery
Blast radius
nearly 600 driverless vehicles under investigation; four known pedestrian incidents including one severe injury
Capability abused
Defense & remediation
Recorded defense
pedestrian detection and crosswalk logic
Guardrail kinds
Physical safety system
Guardrail outcome
Silent
Fix shipped
Cruise operations suspended nationwide the following month
Taxonomy & references
Curator notes
Fleet-level record rather than a single crash: the regulator's finding is that the behaviour was systematic across ~600 agents, not one bad episode. Distinct from the 2023-10-02 pedestrian drag, which is recorded separately. AI Incident Database incident 596.
Supporting sources
Cite as
Enkrypt AI. "NHTSA opened a defect probe into ~600 Cruise driverless cars for failing to exercise caution around pedestrians and crosswalks". AIR, AIR-2023-0024. https://agentincident.vercel.app/i/AIR-2023-0024/ (accessed 2026-10-19).