Tool Allowlisting Drove Agent Attack Success to Zero—but Did Not Make the Agents Safe
A new preprint reports zero attack success in many tests of tool-integrated agents, but the result is more useful—and narrower—than the headline number suggests. The strongest defense did not teach a model to recognize every malicious instruction. It restored a previously trusted tool list before planning, making the benchmark’s attacker-specific tools unavailable.
The study, Universal Defenses for Tool-Integrated LLM Agents Against Adversarial Attacks, evaluates direct and indirect prompt injection, memory poisoning, and a Plan-of-Thought backdoor across four open models and GPT-family models. Its “Normal Tool Recalling” defense frequently reduced attack success rate (ASR) to 0%. Yet original-task success rate (OTSR) sometimes remained low or fell sharply, and the evaluation used static simulated tools rather than consequential external systems.
The practical result is still important. A trusted capability boundary can outperform another layer of model persuasion. But “the forbidden tool could not be selected” is not equivalent to “the agent resisted prompt injection,” and defenders should not merge those claims.
The strongest defense is a capability allowlist
The paper tests two tool controls. Attacker Tool Filtering (ATF) embeds tool descriptions and applies an Isolation Forest to remove semantic outliers. Normal Tool Recalling (NTR) saves the original benign toolset, then replaces the agent’s current toolset with that trusted copy immediately before planning. The researchers combine either control with structured planning, GPT-4o-mini paraphrasing, and model self-reflection.
NTR is the stronger and more deterministic design. In the published implementation, it filters the available tools by membership in the benchmark’s normal_tools mapping. The benchmark defines attack success by whether an attacker goal associated with an injected attacker tool appears in the agent’s messages. Once that separate tool is removed, the measured attack path is structurally unavailable.
That is not a weakness in allowlisting. It is the reason allowlisting works. It does, however, define the scope of the zero: NTR proves the value of keeping untrusted tools out of the execution plane. It does not show that the model can safely use an allowed email, browser, shell, database, or file tool when hostile content redirects a legitimate capability toward the wrong destination.
Zero ASR often coexisted with weak task completion
The paper uses Agent Security Bench (ASB), which contains 10 domain-specific agents and 400 tasks per agent, split between aggressive and non-aggressive attacker tools. ASR measures execution of the attacker goal; OTSR measures completion of the original task. Those two numbers must be read together.
For GPT-4 under indirect prompt injection, the no-defense setting produced 44.56% ASR and 37.75% OTSR. The full NTR combination reduced ASR to 0%, but OTSR fell to 10.06%. Under the backdoor attack, GPT-4 moved from 58% ASR and 92.8% OTSR to 0% ASR and 18.7% OTSR. That is strong containment paired with a large availability and usefulness cost.
The pattern was not uniform. Under direct prompt injection, GPT-4’s NTR combination reduced ASR from 91.56% to 0% while raising OTSR from 0.44% to 11.62%. Several smaller open models improved on both metrics. The paper correctly describes the effect as model- and attack-dependent rather than claiming one prompt stack preserves utility everywhere.
GPT-5 received a narrower appendix evaluation because of time limits: one context-ignoring variant for direct and indirect injection, one combined memory attack, and one non-word backdoor trigger. NTR reached 0% ASR in all four rows, while OTSR ranged from 0.75% to 9.5% across the NTR configurations shown. Those results support testing capability restoration on newer models, but they are not a broad adaptive red-team result.
The benchmark measures simulated traces, not production side effects
The repository makes the evaluation boundary unusually clear. Its tools are simulated objects whose run() method returns a predefined “expected achievement” string. Attack success is detected by searching message content for the attacker-goal string. Original-task success is detected by searching messages for every normal tool’s expected-achievement string.
This gives researchers a repeatable way to compare many model and defense combinations. It does not test whether an agent sent a real message, changed a cloud role, wrote the intended file correctly, recovered from a partial failure, or caused a side effect through a second allowed tool. String-presence scoring can also count a goal mentioned in reasoning or logs without establishing that a production action completed.
The authors acknowledge the larger limitation: they did not evaluate dynamic environments and identify real-world applications and more complex tool ecosystems as future work. Teams should therefore treat the reported rates as evidence about this ASB harness—not as a deployment-grade estimate of prompt-injection risk.
Semantic anomaly filtering has a weaker trust model
ATF assumes an injected tool’s description will be semantically unusual relative to the legitimate set. The code embeds descriptions with all-MiniLM-L6-v2 and runs an Isolation Forest with a 0.33 contamination rate. It then removes the detected outliers.
This may help when an attacker adds an obviously unrelated capability. It is less reliable when the malicious tool imitates the name and description of an approved one, when legitimate tools cover diverse domains, or when a small toolset makes a fixed expected-outlier fraction destructive. The paper’s tables reflect that uncertainty: ATF sometimes lowered ASR, but results varied by model and attack and occasionally trailed simpler baselines.
For production admission, provenance is a stronger signal than semantic normality. A tool should be authorized because its publisher, package, endpoint, schema, version, requested scopes, and integrity are approved—not because its description resembles neighboring tools. Semantic detection is useful telemetry; it is not identity.
The released code is evidence, but not a turnkey reproduction
The paper links a public repository with the ASB-derived harness, task data, simulated tools, attack launchers, and the NTR and ATF implementations. That is enough to inspect the control placement and scoring logic. The release does not include result logs, and its supplied YAML configurations enumerate GPT-4o-mini and GPT-3.5 rather than the paper’s GPT-4 and GPT-5 evaluations.
A fresh reproduction also needs dependency repair. The published requirements.txt omits packages imported by the harness, including the sentence-transformer and Isolation Forest libraries used by ATF, plus several orchestration dependencies. Model endpoints are mutable, and no response corpus or complete run manifest is included. The numerical tables should therefore be attributed to the preprint, not presented as independently reproduced results from the repository.
This does not invalidate the architecture finding. NTR’s core operation is visible and straightforward. It limits how strongly the surrounding model comparisons and exact percentages can be independently audited from the current artifact.
What defensive engineering teams should take from the study
- Keep the approved capability set outside model-writable state. Store tool identity, versions, schemas, scopes, and destinations in a trusted policy layer. Reconstruct the effective set before every planning and execution step.
- Authorize tool plus arguments. A safe tool can still perform an unsafe action. Bind approval to the operation, principal, object, destination, data class, and time window—not only the tool name.
- Recheck at execution. Planning-time filtering is insufficient if a later observation can alter arguments, swap a tool endpoint, change MCP metadata, or invoke a capability through another route.
- Separate admission from anomaly detection. Use signatures, attestations, pinned identities, and explicit policy for admission. Use semantic outlier scores to investigate changes, not to establish trust.
- Measure secure utility. Report attack success, benign completion, refusal, latency, cost, and false-positive denial together. A 0% ASR system that completes almost no legitimate work is contained but not operational.
- Test allowed-tool abuse. Red-team exfiltration through approved messaging and storage tools, confused-deputy access with valid credentials, destination substitution, argument smuggling, chained tools, and delayed side effects.
- Use real state transitions in evaluation. Score observable sandbox effects and invariant violations rather than relying only on model text or expected strings.
- Preserve reproducibility artifacts. Pin model and harness versions, publish run configurations and raw traces, record evaluator logic, and rerun after tool, policy, prompt, or provider changes.
The study’s best lesson is architectural: a model should not decide which capabilities are trusted. Restoring an approved toolset before planning is a useful control because it removes authority rather than asking the same model to reason its way out of hostile instructions. The next step is to carry that boundary through execution—where approved tools, attacker-controlled arguments, real credentials, and irreversible side effects meet.
This extends the site’s prompt-injection defense playbook and complements GuardianAgentBench’s runtime-guardrail results: model-facing defenses can reduce attacks, but deterministic authorization and consequence-aware evaluation are the controls that turn a promising benchmark result into a defensible system.
Sources: