Safety failure
Realized harm
Copilot-generated training scenario named a real sexual-harassment complainant, shown on screen at her former workplace
AIR-2024-0036 AIR record · 2024-08-copilot-bunbury-prison-harassment-victim-name
A real party was affected. This incident caused actual consequences, not a demonstrated capability.
From the supporting source
One scenario included a character called Bronwyn Hendry, the name of a real former employee.abc.net.au
The record
Disclosed
2024-08-21
Occurred
2024-08
Vendor
Microsoft / Psychosocial Leadership (deployer)
Product
Microsoft Copilot
Risk domain
Privacy › Sensitive data exposure
Also touches
Human manipulation
System type
Generative model
Who acted
Human acted on output (contested) — the model generated the scenario; an operator displayed it on screen
Attack surface
Enterprise assistant
Autonomy level
Assistant
Initial vector
None autonomous action
Impact type
Privacy violation, Reputational harm
Confidence
High
Submitted by
EAI Internal
Crosswalk
- OWASP Agentic Top 10 (2026)
- ASI06 – Memory & Context Poisoning
- MITRE ATLAS
- AML.T0057 – LLM Data Leakage
- NIST AI RMF
- MEASURE 2.10
- EU AI Act
- Article 10
- ISO/IEC
- TR 24028; 23894
- AIUC-1
- A. Data & Privacy
- Scope
- tool misuse
- Mitigation
- Use data anonymization; Implement log sanitization to scrub PII.
- Taxonomy
- Enkrypt AI agent risk taxonomy · Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery
Blast radius
one named individual, re-identified in front of colleagues at Bunbury Regional Prison
Capability abused
Defense & remediation
Recorded defense
none; the trainer believed the scenario was fictional
Guardrail kinds
None
Guardrail outcome
Not applicable
Fix shipped
unknown
Taxonomy & references
Curator notes
The operator's assumption — that a generated 'fictional' example contains no real people — is the failure. Asking a model for synthetic examples of a scenario it has seen reported can return the reported instance. Trainer Charlotte Ingham said she walked in thinking she had a fictional scenario. AI Incident Database incident 773.
Supporting sources
Cite as
Enkrypt AI. "Copilot-generated training scenario named a real sexual-harassment complainant, shown on screen at her former workplace". AIR, AIR-2024-0036. https://agentincident.vercel.app/i/AIR-2024-0036/ (accessed 2026-10-19).