Safety failure
Realized harm
A Tesla Model 3 the driver said was in driver-assistance mode left the road in Katy, Texas and killed a woman inside her home
AIR-2026-0138 AIR record · 2026-06-tesla-katy-texas-fatal-home-crash
A real party was affected. This incident caused actual consequences, not a demonstrated capability.
From the supporting source
The driver, Michael Butler, was in a Tesla Model 3 about 8 p.m. local time and operating the car "with an automated driving assistance system," the Harris County Sheriff's Office said in a statement on Saturday.nytimes.com
The record
Disclosed
2026-06-21
Occurred
2026-06-19
Vendor
Tesla
Product
Tesla driver-assistance system
Risk domain
Agent behaviour › Unsafe actuation
System type
Classical autonomy (no generative model)
Who acted
Agent acted (contested) — the driver-assistance system left the road and killed a woman
Attack surface
Autonomous operations
Autonomy level
Semi-autonomous
Initial vector
None autonomous action
Impact type
Physical safety
Confidence
High
Submitted by
EAI Internal
Crosswalk
- OWASP Agentic Top 10 (2026)
- ASI01 – Agent Goal Hijack
- MITRE ATLAS
- AML.T0048 – External Harms
- NIST AI RMF
- MEASURE 2.6; MANAGE 1.3
- EU AI Act
- Annex III §1(c)
- ISO/IEC
- TR 24028; 24029-1; 23894
- AIUC-1
- C. Safety
- Scope
- agent misuse
- Mitigation
- Require confirmation for destructive actions; Implement dry-run modes; Use layered safety controls.
- Taxonomy
- Enkrypt AI agent risk taxonomy · Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery
Blast radius
one resident, Martha Avila, killed inside her home; Harris County and NHTSA investigations opened
Capability abused
Defense & remediation
Recorded defense
lane keeping and driver monitoring
Guardrail kinds
Human supervision
Guardrail outcome
Silent
Fix shipped
none; investigations ongoing
Taxonomy & references
Curator notes
ATTRIBUTION DISPUTED. The driver told investigators assistance was engaged; Tesla's Autopilot vice president said he had overridden self-driving by fully depressing the accelerator. Both can be true, and that is the point worth recording: 'the human overrode it' and 'the system was driving' describe the same seconds, and a design where flooring the pedal silently transfers full authority mid-manoeuvre makes the accountability question unanswerable after the fact. A bystander in her own home was killed; investigations are unresolved. AI Incident Database incident 1552.
Supporting sources
Cite as
Enkrypt AI. "A Tesla Model 3 the driver said was in driver-assistance mode left the road in Katy, Texas and killed a woman inside her home". AIR, AIR-2026-0138. https://agentincident.vercel.app/i/AIR-2026-0138/ (accessed 2026-10-19).