al-ice.ai

Fiddler AI — AI Coding Agent Threat Models and Controls

Security by al-ice.ai Editorial

AI relevance: Coding agents (Claude Code, Codex, Copilot) now operate with repo-write, shell, and cloud-API access — Fiddler AI's analysis maps the combined LLM + code-execution attack surface and proposes runtime guardrails that traditional SAST tools cannot cover.

Key Findings

  • A 2025 study of 7,700+ AI-generated GitHub files found 4,241 CWE instances across 77 vulnerability types — Python files hit 16–18% vulnerability rates. These are not novel patterns; they are decade-old mistakes that SAST tools miss because they were calibrated for human-written code.
  • Coding agents combine two risk categories: LLM-native vulnerabilities (prompt injection, hallucination, memory poisoning) and code-execution vulnerabilities (RCE, privilege escalation, sandbox escape). Traditional AppSec tooling addresses neither combination.
  • Indirect prompt injection is the highest-severity vector: a poisoned GitHub issue comment can instruct an agent to add a backdoor during a routine bug fix. The agent reads the comment as legitimate context and follows the hidden instruction.
  • Supply-chain poisoning via agents: autonomous tools pull packages from public registries, enabling typosquatting and dependency confusion at machine speed — an agent might import requests-toolkit instead of requests without human review.
  • MCP server spoofing introduces a new tool-abuse path: agents connect to external services via Model Context Protocol, and malicious or compromised MCP servers can present legitimate-looking tools that perform unauthorized actions including data exfiltration.
  • Memory poisoning is unique to agents with persistent context: an attacker poisons session memory in step one, then exploits the corrupted context in step ten — a vector that stateless assistants do not expose.
  • Credential exposure compounds across sessions: agents may embed API keys in generated code, log secrets in error messages, or commit plaintext passwords to public repos where automated scrapers collect them within minutes.

Why It Matters

Development teams are adopting coding agents faster than security teams can assess them. The autonomy-without-oversight model means a single poisoned input can cascade through an entire CI/CD pipeline before a human reviews anything. Traditional SAST scanners were optimized for human code patterns — they systematically miss agent-generated vulnerabilities because the structural fingerprints differ.

What To Do

  • Treat all agent-generated code as untrusted input requiring specialized validation, not developer-equivalent output passing through standard review gates.
  • Implement runtime guardrails: sandbox agent shell access, enforce allowlists for package registries, and monitor for credential patterns in agent output before it reaches production.
  • Extend lineage tracking to cover the model, the agent session, and the MCP tools it invokes — not just the final artifact.
  • Deploy agent-specific SAST rules calibrated for AI-generated code patterns, or use LLM-as-judge security scanners that understand agent output semantics.

Sources