The Attack Rewrites Itself: Self-Evolving Poison Outruns Static Defenses in Search Agents

2026-11-04 AI Security Research by al-ice.ai Editorial

Submitted 2 November 2026 to arXiv (2611.03754, accepted at NeurIPS 2026) by researchers at Nanyang Technological University and Alibaba Group, The Web Doesn’t Sit Still: Adversarial Self-Evolving Attacks on Search Agents argues that the way we measure poisoning attacks on search agents is broken in a specific way: every baseline is a static, human-crafted template — swap an entity here, optimize an injection there — while a real adversary adapts. The authors replace the fixed attack with an adversarial self-evolution framework: a bi-level optimization in which an attacker’s strategy pool evolves against a defense that updates in turn, so each round’s poison is bred against the previous round’s filter. Across three multi-hop QA benchmarks and five frontier LLMs, the evolved strategies reach a mean total interference rate of 64.2 to 74.6 percent — more than triple the rule-based baseline — and stay above 60 percent even after the defense synchronizes against them. The defense the paper proposes as the stringent setting is the only one that meaningfully dents the attack, and it does so precisely because it knows the attack pool. Static tests, the paper’s title warns, cannot catch a moving attack.

CREO: contrastive signals breed the strategy pool

The core method, Contrastive Rollout Evolutionary Optimization (CREO), treats attack strategies as a population that undergoes directional mutation, crossover recombination, and selection driven by contrastive evolutionary signals — successful and failed rollouts are contrasted so the pool learns which patterns penetrate, rather than a human guessing. The pool is capped at 10 individuals, seeded by a contrastive initialization (Round 0) and evolved over three rounds. Evaluation runs under two complementary settings: Evo, where the attack pool evolves against the previous round’s static defense (the attacker’s advantage), and Sync, where defenses update synchronously with the current attack pool (the hardest test). To keep evolution honest, everything runs inside a terabyte-scale controlled sandbox with noise-free trace capture, and the authors add fine-grained cognitive metrics that go beyond right-or-wrong answers: rigorous adversarial accuracy (correct answer and reasoning fully free of interference), a deliberately strict trace-level success rate that counts an attack only when the agent explicitly incorporates a forged document as valid evidence, and a novel adversarial paralysis rate for the case where the agent does not get hijacked but stalls — their sum is the total interference rate. The test bed is HotpotQA, 2WikiMultihopQA and MuSiQue (100 training and 100 test queries each) with Qwen-Turbo, Qwen3-Max, DeepSeek-V3.2, Kimi-K2.5 and GLM-5.1 as target search agents, benchmarked against two paradigms: rule-based tampering (FactSwap, substituting answer entities with type-consistent counterparts) and optimization-based poisoning (AuthChain, jointly optimized for retrievability and deceptiveness).

Evolved poison wins, static filters barely notice

The headline numbers are unambiguous. CREO’s mean Evo total interference of 64.2–74.6 percent and mean Sync of 60.5–64.9 percent compare with 20.9 / 18.1 percent for FactSwap and 55.7 / 37.5 percent for AuthChain — and CREO’s Sync figure, measured after the defense adapts, still beats AuthChain’s Evo figure, measured before it does. Answer accuracy falls by 18 to 65 percentage points across all six model–dataset combinations reported, with the worst collapses on MuSiQue: Qwen3-Max from 90 percent to 25 percent, GLM-5.1 from 82 percent to 30 percent. The defense analysis is the part practitioners should sit with. Keyword and perplexity filters provide negligible protection, with interference rates near the no-defense baseline — the evolved forgeries are semantically fluent and contextually coherent, so pattern-matching and anomaly scores slide off. RobustRAG’s isolate-and-aggregate strategy achieves only moderate suppression. Only the paper’s own strategy-aware CREO Defense, which defends with maximal information about the attack pool, yields substantial interference reduction (53 percent on Qwen-Turbo, 17 percent on GLM-5.1) — and the attack still lands. Two methodological details strengthen the claim: after three rounds the strategy pool is genuinely heterogeneous — identity obfuscation, multi-source contradiction construction, multi-hop reasoning poisoning, with the most sophisticated patterns emerging only in later rounds — and the LLM-as-judge scoring was validated against 100 stratified traces with 96–97 percent agreement and Cohen’s κ above 0.94 across two human annotators, GPT-5.6 Sol and Claude fable-5. This is the mirror image of October’s self-evolving defense coverage — that paper evolved the defender; this one evolves the attacker, and the attacker is currently ahead.

Evaluate against the attacker you will actually meet

The paper’s limits are stated plainly and worth repeating: evolution happens inside a controlled sandbox, not the live web, and retrieval operates exclusively within it — the numbers describe what an adaptive adversary can achieve once retrieved, not a measured in-the-wild campaign. The defensive prescription follows directly. Benchmark search agents against adaptive, evolving attacks — not just fixed injection templates, because the gap between FactSwap’s 21 percent and CREO’s 65-plus percent is the gap between your test suite and your adversary. Retire keyword and perplexity filters as primary controls against fluent forgeries; they fail open against exactly the attacks that now evolve. Invest in isolate-and-aggregate architectures and strategy-aware defenses, while understanding that even the best tested here only dented the attack. And keep this paper next to its siblings: our Step-Zero coverage showed harm arriving before the model ever looks, our ETUDE coverage showed trusted tools decaying after updates, and the Radware census and Log-To-Leak work showed injections already hiding in the open. CREO adds the dynamic: the poison does not sit still either.

Our verification was read-only. We read the full arXiv abstract page and the experimental HTML text first-hand (submitted 2 November 2026; Geyuan Wang, Shunyu Liu, Siyuan Liang, Chenyang Lyu, Dacheng Tao corresponding, NTU and Alibaba; accepted at NeurIPS 2026; CREO bi-level formulation with contrastive rollout signals, pool cap 10, Round 0 plus three rounds, Evo and Sync settings; terabyte-scale sandbox; Acc, strict trace-level ASR, APR and TASR metric definitions; HotpotQA, 2WikiMultihopQA and MuSiQue with 100 training and 100 test queries each; Qwen-Turbo, Qwen3-Max, DeepSeek-V3.2, Kimi-K2.5 and GLM-5.1 targets; FactSwap and AuthChain baseline definitions; 64.2–74.6 percent Evo and 60.5–64.9 percent Sync TASR against 20.9/18.1 and 55.7/37.5; 18–65 point accuracy reductions with MuSiQue Qwen3-Max 90-to-25 and GLM-5.1 82-to-30; keyword, perplexity, RobustRAG and CREO Defense outcomes with 53 and 17 percent reductions; strategy-pool diversity and late-round sophistication; judge validation 96–97 percent with kappa above 0.94). Table-extracted figures are the authors’ reported results as stated there. We trained no model, ran no agent, poisoned no retrieval, and contacted no one. Brave Search was used for discovery only.

Sources: