Google’s Bug-Hunting Agent Found 500 XSS Flaws — and Two in the Code Built Safely
Google’s Product Security team published details of PageBreak on September 24 — an internal AI agent that tests Google’s own first-party web applications. The headline figure is that it has uncovered over 500 cross-site scripting vulnerabilities, including on what the post calls sensitive domains. The project began as a pilot in November 2025 and became a full effort in January 2026; it is model-flexible but runs mostly on Gemini models, specifically naming Gemini 3.1 Pro and Gemini 3.5 Flash.
Five hundred XSS bugs is the number that will travel. Two other things in the post are more useful to anyone building or buying this kind of tooling.
The problem the post names first is not vulnerabilities
Author Michał Bentkowski opens on a bottleneck rather than a capability. LLM-based scanning, he writes, has introduced “a significant operational bottleneck: noise” — security teams “overwhelmed as a significant portion of the candidate reports they receive is ‘AI slop’ — noisy, unverified hypotheses or false positives generated by LLMs acting as static code analyzers.” Distinguishing a real flaw from “a convincing hallucination” now “often increas[es] the burden on product teams, rather than reducing it.”
That framing is worth sitting with, because it is a vendor of AI security tooling stating plainly that the default application of LLMs to vulnerability discovery makes the receiving team’s job worse. It matches what maintainers of large open-source projects have been reporting all year about AI-generated bug reports. The interesting design question for PageBreak was therefore not “can a model find bugs” but “how do we stop shipping the model’s opinions to engineers.”
The answer is that the model does not get a vote
PageBreak’s architecture decision, made early, was to “prioritize deterministic validation.” When the agent forms a hypothesis, it hands that hypothesis to a suite of specialized, non-AI-written validators that execute a real payload against a running environment. The post enumerates them by vulnerability class:
- XSS — inject a specific JavaScript payload, load the URL in a rendering harness, and monitor the execution context to detect whether the script actually ran.
- SQL injection — inject payloads and verify via output and/or timing.
- Path traversal — create a file in a world-readable location and check whether the application can read it back.
- RCE — confirm execution through a sleep delay, a write to a world-writable location, or a triggered outbound DNS/HTTP request.
- SSRF — detect that an outbound request to an internal service was actually made.
The clearest sentence in the post is the one describing what this buys: the validator’s job “is not to offer another opinion about the code.” Every one of those checks is a side effect an attacker could observe — a script that executed, a file that exists, a DNS query that arrived. None of them can be produced by a model being persuasive. Google reports this yields “a near-zero false positive rate.”
This is the same architectural move as CodeMender, Google’s patching agent, which validates its fixes against a reproducing exploit rather than against its own confidence — and the post confirms the two projects now collaborate directly, with the stated goal that “product team involvement is ultimately reduced to simply validating proposed fixes.” It is also the inverse of the failure mode we keep documenting on the offensive side, where an agent’s output is trusted because it is fluent. Here, fluency earns nothing; a payload that runs earns everything.
Google is explicit about the cost. The validators “are currently lacking in capabilities to cover all vulnerability types or complex scenarios,” risking false negatives. So non-deterministic findings are kept, but used only internally — to seed later runs, to identify where validators fall short, and to let the agent report what capability or environment access it was missing. “Crucially,” the post says, “we do not send these unverified candidates to product teams.” The unverified output is treated as telemetry about the tool, not as findings about the product.
The number that matters is two
The most defensively significant paragraph is the shortest. Google ran the same agent against applications built on the high-assurance web framework it described in its 2025 Secure by Design: Google’s Blueprint for a High-Assurance Web Framework post — a framework designed to eliminate classes of web vulnerability structurally rather than by review.
As of September 4, 2026, PageBreak found two XSS vulnerabilities across hundreds of applications built on those frameworks, and both were “limited to internal applications or debug endpoints with hardening gaps.”
Same scanner. Same organization. Same year. Over 500 findings in the general first-party population; two in the population built on a framework that makes the bug class hard to express. That is a controlled comparison of a kind the industry almost never gets to publish, and it points the opposite direction from where most security budget is currently moving. An agentic scanner that verifies its own exploits is a genuinely strong measuring instrument — and what it measured most convincingly is that the vulnerabilities were prevented upstream by design, not found downstream by scanning.
The honest caveat: this is Google reporting on Google’s framework, with no external replication, and the comparison populations are not necessarily matched for age, complexity, or exposure. But the direction of the result is hard to explain away. A scanner with a near-zero false-positive rate running at massive scale found almost nothing in the safe-by-construction population.
Why this does not transfer cleanly
Google names three advantages that are structural to Google, and buyers evaluating agentic scanners should read them as caveats rather than features.
The monorepo. With billions of lines in a single repository, the agent can “follow complete execution paths end-to-end without ever having to consult external resources” — including the configurations of the services involved. Most organizations cannot give an agent a closed world; their code, infrastructure, and third-party dependencies live in separate places with separate access. Incomplete context is where hypotheses become hallucinations.
Security signals from live traffic. PageBreak maps HTTP paths to specific lines of source code using data extracted from production traffic, which is how it discovers attack surface rather than guessing at it.
Existing scanning infrastructure. This is the quiet one. PageBreak repurposes Google’s long-established web scanners — including one that can authenticate to nearly every Google web application. Authentication into internal surfaces is precisely what makes the deep findings reachable, and it is years of unglamorous engineering that has nothing to do with the model. The post says so directly: PageBreak can evaluate internal sites “that would otherwise be difficult to scan or would even be inaccessible to most security engineers.”
The post adds one more operational detail that quietly undercuts any “the agent just does it” reading: because models “can still deviate into unproductive paths,” Google executes agents “with identical seeds across numerous iterations, maximizing the probability of discovering the correct exploit trajectory.” The agent is not reliably finding these bugs. It is being run many times until one trajectory lands, and a deterministic validator decides which one counted.
What defenders should take from this
- Judge agentic security tools by their validator, not their model. The question to ask a vendor is what observable side effect confirms a finding before it reaches your queue — a script that executed, a file that exists, a request that arrived. “The model is confident” is not an answer. This is the single most portable idea in the post.
- Measure the false-positive burden as a first-class cost. Google’s stated reason for the whole architecture is that unverified LLM findings increase the burden on product teams. If a tool’s output still requires a human to determine whether each finding is real, it has moved work, not removed it.
- Route unverified findings to tool development, not to engineers. Google keeps its non-deterministic findings and uses them to seed future runs and to identify missing validators. That is a reusable pattern for any team running an internal scanning agent.
- Note that scanning capability and elimination capability are not substitutes. The two-versus-500 result argues for spending on frameworks that make the bug class unrepresentable, and using the agent as the instrument that proves it worked.
- Assume the transferable part is smaller than the headline. Monorepo-wide context, traffic-derived path-to-source mapping, and an authenticating scanner fleet are doing substantial work here. Reproducing the results without them is a different project.
PageBreak is a genuinely strong result, and it is strong for an unfashionable reason. The advance is not that a Gemini model is better at spotting injection points; it is that Google built a wall of deterministic, hand-written checks between the model’s hypotheses and the humans who would have to act on them, then declined to let anything through that could not prove itself by execution. The same discipline that makes an agentic scanner trustworthy — enforcement outside the model, verified by side effect — is the one missing from nearly every agent framework advisory we cover on the vulnerability side. It is easier to build when the thing being constrained is your own tool rather than an attacker’s input. It is the same principle either way.
Sources: