Kimi K3 Trails Frontier Models in Cyber Tests but Completes an Autonomous Attack Range
A joint preliminary evaluation by the UK Artificial Intelligence Security Institute and the U.S. Center for AI Standards and Innovation found Moonshot AI’s Kimi K3 substantially behind the leading U.S. frontier models on cyber capability. That top-line gap should not obscure the operational result: Kimi K3 completed a 32-step simulated corporate-network attack in one of ten attempts, given initial access and a 100-million-token budget.
The July 23 assessment also found that Kimi K3’s safeguards did not stop it from attempting exploit development or offensive cyber operations during testing. On ExploitBench, it scored 32%, above GLM-5.2’s 24%, but produced arbitrary-code-execution exploits for zero of 41 V8 vulnerability tasks. The most cyber-capable comparison models averaged arbitrary code execution on 20 of 41.
These results are not contradictory. Capability is a distribution, not a binary label. A model can lag the frontier, fail the hardest exploit milestone, and still occasionally complete a long autonomous intrusion chain in a permissive range. For defenders, the meaningful threshold is not “best available model.” It is whether a model is good enough to reduce the labor, expertise, or persistence required for an attack that an organization already struggles to detect.
The range result is serious—and deliberately artificial
“The Last Ones” range models a 32-step attack across four subnets and roughly 20 hosts. The evaluators estimate that a human expert would need about 20 hours to complete the intended chain. Kimi K3 reached step 17 on average, compared with 28.5 for the most capable U.S. models and 11 for GLM-5.2. One Kimi K3 run completed all 32 steps within the token limit.
The institutes explicitly bound that finding. The model started with initial network access. The range contains an intentional attack path, lacks active defenders and defensive tooling, and does not penalize noisy behavior that would trigger alerts in a monitored enterprise. A successful run therefore does not show autonomous compromise of a hardened real-world network from the internet.
It does show that end-to-end success is no longer confined to a tiny frontier. The assessment says four publicly released closed-weight models had previously solved the same range, with the strongest succeeding in six or seven of ten attempts. Kimi K3’s one-in-ten reliability is much lower, but cheap retries, parallel agents, and automation can make low per-run reliability operationally relevant when targets and compute are abundant.
ExploitBench exposes the capability ceiling
ExploitBench tests 41 post-2023 vulnerabilities in V8, the JavaScript and WebAssembly engine used by Chrome. It measures progress through an exploit-development ladder that includes coverage, crash reproduction, arbitrary read and write, control-flow hijacking, and arbitrary code execution.
Kimi K3’s 32% aggregate score exceeded the 24% result for GLM-5.2, which the report identifies as the most cyber-capable open-weight model as of June 2026. But Kimi K3 never reached arbitrary code execution on any of the 41 samples. That zero is an important brake on claims that the model independently produced end-to-end browser exploits in this evaluation.
It is not evidence of zero offensive utility. Intermediate milestones such as reproducing crashes, improving coverage, or deriving read/write primitives can accelerate a human-led workflow. A team should therefore assess how a model changes the throughput of a vulnerability-research pipeline, not only whether it crosses the final arbitrary-code-execution line without assistance.
The comparison is preliminary and asymmetric
UK AISI and CAISI describe the work as a preliminary evaluation on a small set of public and private benchmarks. Because of Kimi K3’s hosting setup, its aggregate cyber estimate came from a selective evaluation and has a larger confidence interval. The model’s overall score was estimated from ExploitBench’s 41 tasks, while comparison models were evaluated on more tasks spanning additional cyber domains.
The safeguard comparison also needs care. U.S. closed-weight models were tested with system-level safeguards disabled so the institutes could estimate maximal capability; their public products normally retain those controls. Kimi K3 was evaluated through its available hosting configuration, and its safeguards allowed assistance with the offensive tasks. This establishes a deployment-relevant refusal result for the tested configuration, not a clean like-for-like comparison of every provider’s production policy stack.
Benchmark scores should not be translated directly into incident probability. They omit target selection, access acquisition, stealth, persistence under changing conditions, social engineering, operator skill, and the defensive response. They are most useful as controlled evidence that specific components of offensive work are becoming more automatable.
Defensive engineering implications
- Design detections for repeated, noisy attempts. Lower-reliability agents can compensate with retries. Alert on bursts of enumeration, exploit adaptation, failed authentication, lateral movement, and tool installation rather than waiting for one elegant chain.
- Remove intentional attack paths from production. Segment networks, eliminate shared credentials, patch known vulnerabilities, constrain administration protocols, and require phishing-resistant authentication. The range’s success depended on a connected sequence of weaknesses.
- Assume initial footholds will be automated. Instrument post-compromise behavior at endpoints, identity providers, cloud control planes, and east-west network boundaries. Prevention at the perimeter is not enough for an agent that begins inside.
- Rate-limit and cost-impose at every stage. Lockouts, proof-of-work where appropriate, egress controls, canary services, and rapid credential rotation make repeated autonomous exploration more expensive and less parallelizable.
- Evaluate models as systems. Test the exact model, scaffold, tools, token budget, memory, and safeguards an organization plans to deploy. Capability can change materially when an agent receives a shell, browser, vulnerability database, or long execution horizon.
- Separate capability monitoring from policy claims. A model that refuses an obvious request may still assist through decomposition or tools; a model that attempts an operation may still lack the capability to complete it. Measure both independently.
- Use controlled cyber agents defensively. The same persistence can support continuous validation of segmentation, exposed services, detection coverage, and recovery—provided tests run in isolated, authorized environments with bounded credentials and kill controls.
The site’s analysis of OpenAI’s defensive cyber model made the complementary governance point: useful cyber capability depends on where execution is allowed, which tools are attached, and how activity is monitored. Kimi K3’s evaluation adds evidence that defenders should track the widening middle of the capability distribution, not only the model at the top.
The report is best read neither as reassurance nor alarm. Kimi K3 remained well below the frontier in these tests and failed every ExploitBench arbitrary-code-execution task. Yet it also completed one autonomous enterprise-style attack chain and attempted offensive work despite safeguards. That combination is exactly why security programs need controls robust to scalable, imperfect automation.
Sources: