Disclosed vulnerability
Bing Chat's system prompt ("Sydney") extracted by prompt injection during limited preview
AIR-2023-0004 AIR record · 2023-02-microsoft-bing-chat-sydney-prompt-leak
From the supporting source
Early testers of Bing Chat successfully used prompt injection to reveal its built-in initial instructions, which contains a list of statements governing ChatGPT's interaction with users.arstechnica.com
The record
Disclosed
2023-02-08
Occurred
2023-02-08
Vendor
Microsoft / OpenAI
Product
Bing Chat (New Bing)
Risk domain
Privacy › Sensitive data exposure
System type
Generative model
Who acted
Elicitation only (contested) — the model was made to emit its own system prompt; no victim, no action
Attack surface
Enterprise assistant
Autonomy level
Assistant
Initial vector
Direct prompt injection untrusted input
Impact type
Information disclosure
Confidence
High
Submitted by
EAI Internal
Crosswalk
- OWASP Agentic Top 10 (2026)
- ASI06 – Memory & Context Poisoning
- MITRE ATLAS
- AML.T0057 – LLM Data Leakage
- NIST AI RMF
- MEASURE 2.10
- EU AI Act
- Article 10
- ISO/IEC
- TR 24028; 23894
- AIUC-1
- A. Data & Privacy
- Scope
- tool misuse
- Mitigation
- Use data anonymization; Implement log sanitization to scrub PII.
- Taxonomy
- Enkrypt AI agent risk taxonomy · Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery
Blast radius
Bing Chat preview users; the confidential system prompt for all sessions
Capability abused
Defense & remediation
Recorded defense
confidentiality instruction in the system prompt
Guardrail kinds
Prompt instruction
Guardrail outcome
Silent
Fix shipped
unknown
Taxonomy & references
Curator notes
Kevin Liu (Stanford) extracted the prompt days after launch. Foundational case: instruction-level confidentiality in an LLM product is not a security boundary. No third-party harm, hence impact_realized false. AI Incident Database incident 473.
Supporting sources
Cite as
Enkrypt AI. "Bing Chat's system prompt ("Sydney") extracted by prompt injection during limited preview". AIR, AIR-2023-0004. https://agentincident.vercel.app/i/AIR-2023-0004/ (accessed 2026-10-19).