In the wild
Realized harm
Google sued the China-based Outsider Enterprise network for using Gemini to mass-produce phishing sites behind 2.5 million scam text messages
AIR-2026-0135 AIR record · 2026-05-google-outsider-enterprise-gemini-phishing-lawsuit
A real party was affected. This incident caused actual consequences, not a demonstrated capability.
From the supporting source
The Chinese group used Gemini, Google's A.I. system, to create hundreds of fake websites mimicking companies like Google and YouTube and government operations like the Postal Service and New York's E-ZPass service for highway tolls, the lawsuit said.cn.nytimes.com
The record
Disclosed
2026-06-12
Occurred
2026-05
Vendor
Google (abused); US consumers (victims)
Product
Gemini
Risk domain
Agent behaviour › Human manipulation
System type
Generative model
Who acted
Human acted on output (contested) — Gemini generated the phishing sites; the network sent 2.5M scam texts
Attack surface
Other
Autonomy level
Assistant
Initial vector
Policy violating use
Impact type
Financial loss, Ecosystem abuse
Confidence
High
Submitted by
EAI Internal
Crosswalk
- OWASP Agentic Top 10 (2026)
- ASI09 – Human-Agent Trust Exploitation
- MITRE ATLAS
- AML.T0054 – LLM Jailbreak
- NIST AI RMF
- MAP 5.1
- EU AI Act
- Recital 45
- ISO/IEC
- TR 24028; 42001; 23894
- AIUC-1
- C. Safety
- Scope
- agent misuse
- Mitigation
- Communicate model limitations; Require human oversight; Watermark AI content; Design for explicit consent.
- Taxonomy
- Enkrypt AI agent risk taxonomy · Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery
Blast radius
thousands of fake sites impersonating Google, YouTube, USPS and E-ZPass; 2.5 million Android messages in a two-week May 2026 window; hundreds of thousands of people scammed
Capability abused
Defense & remediation
Recorded defense
Gemini usage policy enforcement
Guardrail kinds
Abuse detection
Guardrail outcome
Silent
Fix shipped
civil suit in SDNY seeking injunction; first coordinated takedown with the FBI and AT&T, T-Mobile and Verizon
Taxonomy & references
Curator notes
Notable as remedy rather than technique: the provider went to court and coordinated with the FBI and the carriers, because account termination alone does not reach a network that shares AI toolkits over Telegram. Google stated AI-driven scams are growing faster than other phishing categories. Allegations from a filed complaint, not proven findings. AI Incident Database incident 1534.
Supporting sources
Cite as
Enkrypt AI. "Google sued the China-based Outsider Enterprise network for using Gemini to mass-produce phishing sites behind 2.5 million scam text messages". AIR, AIR-2026-0135. https://agentincident.vercel.app/i/AIR-2026-0135/ (accessed 2026-10-19).