AIR-2026-0153 · An unoccupied Waymo drove over a lit firework in San Francisco and caught fire

An unoccupied Waymo drove over a lit firework in San Francisco and caught fire

AIR-2026-0153 AIR record · 2026-07-waymo-firework-vehicle-fire

A real party was affected. This incident caused actual consequences, not a demonstrated capability.

From the supporting source

"An unoccupied vehicle drove over a small firework in the street and caught fire," a Waymo spokesperson said Sundaysfchronicle.com

The record

Disclosed

2026-07-05

Occurred

2026-07-04

Vendor

Waymo (Alphabet)

Product

Waymo Driver

Risk domain

Agent behaviour › Unsafe actuation

System type

Classical autonomy (no generative model)

Who acted

Agent acted — the vehicle drove over the lit firework

Attack surface

Autonomous operations

Autonomy level

Fully autonomous

Initial vector

None autonomous action

Impact type

Property damage

Confidence

High

Submitted by

EAI Internal

Crosswalk

OWASP Agentic Top 10 (2026): ASI01 – Agent Goal Hijack
MITRE ATLAS: AML.T0048 – External Harms
NIST AI RMF: MEASURE 2.6; MANAGE 1.3
EU AI Act: Annex III §1(c)
ISO/IEC: TR 24028; 24029-1; 23894
AIUC-1: C. Safety
Scope: agent misuse
Mitigation: Require confirmation for destructive actions; Implement dry-run modes; Use layered safety controls.
Taxonomy: Enkrypt AI agent risk taxonomy · Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery

Blast radius

one vehicle destroyed near the 1200 block of Connecticut Street; no occupants, no injuries

Capability abused

autonomous_drivingroad_hazard_recognition

Defense & remediation

Recorded defense

road-debris detection

Guardrail kinds

Unknown

Guardrail outcome

Silent

Fix shipped

none disclosed

Taxonomy & references

no_adversaryunmodelled_hazardlong_tail_scenariophysical_world

Curator notes

Minor in consequence, kept as a clean long-tail example: a lit firework is a road object a human driver recognises instantly as do-not-drive-over, and it is almost certainly absent from the agent's training distribution. The corpus's physical-autonomy records are dominated by exactly this category — situations obvious to any local human and unmodelled by the system. AI Incident Database incident 1583.

Cite as

Enkrypt AI. "An unoccupied Waymo drove over a lit firework in San Francisco and caught fire". AIR, AIR-2026-0153. https://agentincident.vercel.app/i/AIR-2026-0153/ (accessed 2026-10-19).