Safety failure
Realized harm
PayPal's generative AI assistant told a customer a transaction had been declined when no such transaction existed
AIR-2024-0025 AIR record · 2024-06-paypal-ai-assistant-false-decline
A real party was affected. This incident caused actual consequences, not a demonstrated capability.
From the supporting source
Given that this happened to me before I typed anything to the chatbot, you can bet it's happening to others.wkiri.com
The record
Disclosed
2024-07-13
Occurred
2024-06-19
Vendor
PayPal
Product
PayPal generative AI assistant (beta)
Risk domain
Agent output quality › Hallucination
Also touches
Uncontrolled resource consumption, Confused deputy
System type
Generative model
Who acted
Agent acted (contested) — the assistant volunteered a false account state unprompted; output-is-the-harm, no tool call
Attack surface
Consumer assistant
Autonomy level
Assistant
Initial vector
None autonomous action
Impact type
Integrity violation, Service disruption
Confidence
High
Submitted by
EAI Internal
Crosswalk
- OWASP Agentic Top 10 (2026)
- ASI08 – Cascading Failures
- MITRE ATLAS
- AML.T0062 – Discover LLM Hallucinations
- NIST AI RMF
- MEASURE 2.5
- EU AI Act
- —
- ISO/IEC
- TR 24028; 24029-1
- AIUC-1
- D. Reliability
- Scope
- agent failure
- Mitigation
- Implement fact-checking via knowledge base/RAG; Tune model temperature/top-p.
- Taxonomy
- Enkrypt AI agent risk taxonomy · Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery
Blast radius
one reported user, but the false alert was volunteered before any user input, so any beta user could receive one
Capability abused
Defense & remediation
Recorded defense
none; the assistant was in beta and labelled as such
Guardrail kinds
None
Guardrail outcome
Not applicable
Fix shipped
unknown
Taxonomy & references
Curator notes
Small in consequence, kept for the shape. The assistant did not answer a question badly — it volunteered a fabricated fact about the user's own account balance state, unprompted, in a financial product where an account state assertion is the thing users act on. Reported by researcher Kiri Wagstaff. The stated efficiency case for the agent inverted: it generated work for the customer and for the human support representative who had to unpick it. AI Incident Database incident 748.
Supporting sources
Cite as
Enkrypt AI. "PayPal's generative AI assistant told a customer a transaction had been declined when no such transaction existed". AIR, AIR-2024-0025. https://agentincident.vercel.app/i/AIR-2024-0025/ (accessed 2026-10-19).