In the wild
Realized harm
Google AI Overviews and ChatGPT surfaced a scammer's cruise-line support number as the official one, and a user who called it was defrauded
AIR-2025-0098 AIR record · 2025-08-google-ai-overviews-fraudulent-hotline
A real party was affected. This incident caused actual consequences, not a demonstrated capability.
From the supporting source
Consumer Alex Rivlin reportedly called the number, spoke with an impostor, and provided his credit card, later incurring $768 in fraudulent charges.washingtonpost.com
The record
Disclosed
2025-08-15
Occurred
2025
Vendor
Google / OpenAI
Product
Google AI Overviews; ChatGPT
Risk domain
Agent behaviour › Unsafe actuation
Also touches
Confused deputy
System type
Generative model
Who acted
Human acted on output — the AI surfaced the scam number; the user called it and was defrauded
Attack surface
Consumer assistant
Autonomy level
Assistant
Initial vector
Search result poisoning
Impact type
Financial loss, Integrity violation
Confidence
High
Submitted by
EAI Internal
Crosswalk
- OWASP Agentic Top 10 (2026)
- ASI01 – Agent Goal Hijack
- MITRE ATLAS
- AML.T0048 – External Harms
- NIST AI RMF
- MEASURE 2.6; MANAGE 1.3
- EU AI Act
- Annex III §1(c)
- ISO/IEC
- TR 24028; 24029-1; 23894
- AIUC-1
- C. Safety
- Scope
- agent misuse
- Mitigation
- Require confirmation for destructive actions; Implement dry-run modes; Use layered safety controls.
- Taxonomy
- Enkrypt AI agent risk taxonomy · Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery
Blast radius
one named victim charged $768; the same poisoned number surfaced for Royal Caribbean, Disney and Princess cruise queries
Capability abused
Defense & remediation
Recorded defense
Google's spam and quality systems for the underlying index
Guardrail kinds
Unknown
Guardrail outcome
Silent
Fix shipped
undisclosed
Taxonomy & references
Curator notes
The clearest realized-harm case for retrieval poisoning in the corpus. Hotline scams are decades old; what changed is that the AI layer stripped the provenance cues — no domain, no ranking, no visible source — and reissued the attacker's number as a direct answer in Google's own voice. Reported by the Washington Post. Verified quote is the AI Incident Database's own summary. AI Incident Database incident 1187.
Supporting sources
Cite as
Enkrypt AI. "Google AI Overviews and ChatGPT surfaced a scammer's cruise-line support number as the official one, and a user who called it was defrauded". AIR, AIR-2025-0098. https://agentincident.vercel.app/i/AIR-2025-0098/ (accessed 2026-10-19).