al-ice.ai

DataDome — MCP prompt injection & tool poisoning defenses

• Category: Security

AI relevance: MCP servers bundle agent tool access (credentials, business APIs, file/network actions) — so prompt injection/tool poisoning can directly drive unauthorized tool invocations and data exfiltration.

  • Threat model: treat MCP as a high-value target because it sits between LLM reasoning and real tools (tokens, data stores, business systems).
  • Prompt injection recap: direct injection comes via user input; indirect injection comes via retrieved content (web pages, tickets, issues, docs) that gets shoved into the agent’s context.
  • Tool poisoning: malicious instructions can hide inside tool metadata (names/descriptions) that the model sees but the UI may truncate or not render at all.
  • Persistence risk: “rug pull” attacks are plausible in agent tool ecosystems: a tool is reviewed/approved once, then its definition changes later.
  • Evidence: the MCPTox benchmark reports high tool-poisoning success rates across multiple agents; notably o1-mini at 72.8% attack success in that evaluation.
  • Alignment gap: safety training isn’t tuned for “legitimate tool use for illegitimate goals” (e.g., reading secrets, sending them out) — so refusals can be rare in practice.
  • Guardrail reality: MCP’s own spec text (“human in the loop to deny tool invocations”) reads like a SHOULD — but for high-risk tools you should treat it as a MUST.
  • Defense posture: no single mitigation wins; you need layered controls across inputs, permissions, tool supply chain, and runtime monitoring.

Why it matters

  • Blast radius is structural: as soon as an agent can browse + call tools, “text” attacks can become actions.
  • Ops teams own the risk: these failures show up as abuse of credentials, egress, and audit gaps — classic incident response territory, just triggered by LLM context.
  • Tool ecosystems are supply chains: MCP servers and tool registries need software-supply-chain style controls (review, version pinning, integrity checks, monitoring for changes).

What to do

  1. Remove ambient authority: issue scoped, short-lived credentials per tool; avoid “service role” tokens shared across tools/sessions.
  2. Constrain execution: sandbox tool runtimes (containers/VMs), restrict filesystem paths, and block outbound network by default (explicit allowlists only).
  3. Govern tool definitions: version-pin and review tool metadata; alert on changes (treat description changes as security-relevant).
  4. Gate risky actions: require explicit user approval for actions that write, send, or export data (email/webhooks/file uploads/CI deploys).
  5. Instrument intent: log tool calls + parameters; detect anomalies like “read secrets → exfil domain” sequences and auto-revoke permissions on trigger.

Sources