Balungpisah — Critical prompt injection and rate-limiting flaws found in LLM Gateway
AI relevance: LLM Gateways are critical infrastructure for agentic workflows; vulnerabilities here can lead to system prompt leakage, tool abuse, and significant financial loss through DoS.
- A community-led security audit of the balungpisah-llm-gateway (ADK-based) identified two critical vulnerabilities.
- Issue #1: No Prompt Injection Protection: The gateway accepts user messages without sanitization, allowing attackers to manipulate system prompts and hijack agent behavior (CVSS 8.5).
- Issue #2: No Rate Limiting: The lack of per-user or per-org throttling makes the system vulnerable to Model DoS and "cost attacks," where automated scripts could drain OpenAI/Anthropic balances in minutes (CVSS 7.5).
- The audit also flagged a high-severity issue regarding the lack of output content filtering, potentially allowing the LLM to return harmful content or PII.
- The findings were mapped against the OWASP LLM Top 10, showing failures in LLM01, LLM02, LLM04, and LLM06.
- Recommendations include implementing the governor crate for rate-limiting and using separate "Verifier" LLMs to validate prompts before processing.
Why it matters
- Production LLM gateways must treat user input as adversarial; without structural defenses, they risk both technical compromise and financial ruin.
What to do
- Implement Rate Limiting: Add token-bucket or fixed-window throttling at the gateway layer immediately.
- Harden System Prompts: Use delimiters and instruction-guarding techniques to prevent users from "ignoring previous instructions."
- Validate Outputs: Run content safety pipelines against LLM responses before delivery to end-users or tools.