In the wild
Realized harm
LLM training crawlers ignoring robots.txt and spoofing browser identity knocked KDE, SourceHut, GNOME and Fedora infrastructure offline
AIR-2025-0013 AIR record · 2025-03-llm-crawlers-foss-infrastructure-dos
A real party was affected. This incident caused actual consequences, not a demonstrated capability.
From the supporting source
Despite configuring standard defensive measures---adjusting robots.txt, blocking known crawler user-agents, and filtering suspicious traffic---Iaso found that AI crawlers continued evading all attempts to stop them, spoofing user-agents and cycling through residential IP addresses as proxies.arstechnica.com
The record
Disclosed
2025-03-17
Occurred
2025-03
Vendor
multiple AI companies
Product
AI training crawlers operated by Alibaba, Amazon and others
Risk domain
Tool misuse › Uncontrolled resource consumption
Also touches
Unsafe actuation
System type
Generative model
Who acted
Agent acted — the crawlers autonomously fetched at a rate that took the infrastructure offline
Attack surface
Autonomous operations
Autonomy level
Fully autonomous
Initial vector
None autonomous action
Impact type
Service disruption, Financial loss
Confidence
High
Submitted by
EAI Internal
Crosswalk
- OWASP Agentic Top 10 (2026)
- ASI02 – Tool Misuse and Exploitation
- MITRE ATLAS
- AML.T0029 – Denial of ML Service
- NIST AI RMF
- MAP 3.2
- EU AI Act
- —
- ISO/IEC
- TR 24028; 42001; 23894
- AIUC-1
- D. Reliability
- Scope
- tool failure
- Mitigation
- Impose rate limits, timeouts, and resource quotas on agent actions.
- Taxonomy
- Enkrypt AI agent risk taxonomy · Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery
Blast radius
KDE GitLab, SourceHut, GNOME and Fedora infrastructure; volunteer-run FOSS hosting generally
Capability abused
Defense & remediation
Recorded defense
robots.txt, user-agent blocking, traffic filtering
Guardrail kinds
Unknown
Guardrail outcome
Silent
Fix shipped
none by the crawler operators; projects deployed proof-of-work challenges (Anubis) and country-level blocks
Taxonomy & references
Curator notes
An autonomous-agent denial of service with no adversarial intent, which is why it belongs here. The agents actively defeated the consent mechanism — spoofing identity and rotating residential proxies is evasion, whatever the operator intended. Victims are volunteer-run projects absorbing the compute bill. Drew DeVault's post title, 'Please stop externalizing your costs directly into my face', names the pattern. AI Incident Database incident 1001.
Supporting sources
Cite as
Enkrypt AI. "LLM training crawlers ignoring robots.txt and spoofing browser identity knocked KDE, SourceHut, GNOME and Fedora infrastructure offline". AIR, AIR-2025-0013. https://agentincident.vercel.app/i/AIR-2025-0013/ (accessed 2026-10-19).