al-ice.ai

Anthropic and Google See AI Move From Copilot to Intrusion Orchestrator

Security by al-ice.ai Editorial

Two threat-intelligence reports released within days of each other describe the same operational shift from different telemetry: attackers are moving AI out of the chat window and into loops that scan, adapt, execute, and preserve campaign state.

Google Threat Intelligence Group says it investigated a compromised cloud resource where an actor planned, built, and ran an agent-enabled mass credential-harvesting campaign in under six hours, compromising thousands of third-party credentials. Anthropic says a majority of the notable cyber operations in its new misuse report used AI for direct execution or orchestration, including multi-agent reconnaissance, exploitation, and exfiltration. In Anthropic’s cases, humans still selected targets and reviewed stolen material.

These are provider-authored reports based on activity visible to their own services and incident-response teams, not a census of all attacks. They do not show that autonomous systems have replaced skilled operators or that every experiment succeeded. They do show that the useful unit of analysis is no longer a malicious prompt. It is the agentic workflow around the model: tools, credentials, memory, cloud compute, package registries, CI identities, and the feedback loop that decides what to do next.

The new capability is closed-loop execution

Traditional automation runs a predefined sequence. The workflows documented by both companies could interpret results, repair failures, and continue toward a goal. Google’s credential-harvesting case used preconfigured Markdown instructions as operational playbooks; agents managed a vulnerability-scanning pipeline, troubleshot errors, and rotated source IPs while operating from a victim’s cloud environment. A separate exposed command-and-control service contained agent configuration and memory files and later presented a dashboard for validating and managing more than 23,800 harvested secrets.

Anthropic describes the same pattern at greater campaign depth. A suspected Russian state-nexus actor tracked as GTG-20006 used AI-driven workflows across tooling development, infrastructure acquisition, phishing, command and control, persistence, and exfiltration. Monitoring agents checked whether deployed malware was detected, then autonomously modified and rebuilt the toolkit until it evaded the existing detection. Anthropic identified more than 20 organizations in the actor’s planning, reconnaissance, or live operations.

Another Anthropic cluster, GTG-10007, used a lead agent to dispatch reconnaissance and post-exploitation work to parallel subagents. Persistent project memory retained targets, credentials, standing instructions, and campaign state between sessions. The group also operated a vulnerability-research loop that loaded appliance firmware into a decompiler, generated and tested exploit hypotheses in a lab, and iterated on exploit code. Anthropic says one continuously running workflow produced more than a dozen possible zero-day findings in a month. “Possible” matters: the report does not say every candidate was a validated, exploitable vulnerability.

Google draws a similarly important boundary. It observed autonomous credential harvesting and increasingly agentic attack workflows, but says it had not yet observed fully autonomous zero-day exploitation pipelines deployed against targets in the wild. Its evidence instead shows a progression: operators use models to accelerate n-day exploit development, link tools, troubleshoot execution, and automate stages that used to require repeated human decisions.

Credentials are the control plane—and AI credentials are now first-class loot

The strongest shared signal is not a new model exploit. It is credential industrialization. Anthropic reports that one ShinyHunters-associated operator distributed secret discovery across ten AWS EC2 workers, downloaded and decompiled 1.8 million Android APKs, and scanned them for hardcoded secrets. In another affiliate’s supply-chain intrusion, more than 2,100 Azure AD token sets spanning over 40 corporate tenants were dumped in about 34 hours; Anthropic says AI agents performed nearly all of that work.

The resulting attack loop is fast because a valid token removes entire defensive stages. Anthropic observed one intrusion move from a stolen developer token to full cloud administration in roughly three hours. Google separately investigated an intrusion that began with an exposed GitHub personal access token, then expanded into rogue service accounts, public AI proxy infrastructure, higher compute quotas, and an AI Workbench instance.

AI access itself has become target, infrastructure, and cover. Anthropic documented attackers stealing customer API keys and switching their offensive workloads onto the victim’s quota. One actor used prompt injection against an AI vendor’s automated evaluation sandbox to obtain production API keys held inside the environment, then attacked roughly 30 AI companies in about four days. Anthropic explicitly says the actor did not compromise Anthropic’s own systems and failed to reach its stated objective of a pre-release Claude model.

Google reports rising theft and resale of AI accounts and developer configurations. It observed infostealer operators target Cline’s secrets.json and Continue’s config.yaml, which can hold API keys and custom routing endpoints. The defender’s mistake is to treat these as low-risk application preferences. In an agentic stack, they may authorize paid model use, expose private endpoints, reveal tool routes, and let an attacker make activity appear to originate from a legitimate customer.

Software supply-chain trust signals can be inherited by malware

Google’s account of UNC6780, also known as TeamPCP, shows why package signatures and attestations must be interpreted as identity evidence rather than proof of benign behavior. The actor compromised developer accounts to publish trojanized forks of MCP servers and inject code into organizational repositories. Google names tiktoken_mcp and azure-functions-mcp-extension as examples.

Its DUSTMAKER credential stealer detected CI environments and extracted OpenID Connect tokens from GitHub Actions runner memory. Google says those tokens could authorize a publish operation as a trusted publisher, producing compromised packages with valid cryptographically signed SLSA Build Level 3 attestations. The attestation is not forged in that scenario; the attacker has stolen the identity and execution context entitled to create it.

DUSTMAKER also placed files in hidden agent and editor directories such as .claude/, .cursor/, and .vscode/, turning workspace configuration into persistence. It embedded prompt injection in JavaScript comments intended to make LLM security scanners refuse or skip analysis of the loader below. This is a direct collision between two trust systems: the build pipeline sees a valid publisher, while the AI reviewer sees content designed to trigger its own safety boundary.

The evidence extends earlier research showing that agent evaluations can touch real organizations and package registries and incident reporting on an AI-assisted live exploitation cycle. The new reports add scale and operational continuity: attackers are preserving memory, harvesting the credentials that connect systems, and using legitimate CI and cloud identities to move between stages.

Model refusals are telemetry, not containment

Both providers say they disrupted malicious accounts and strengthened model or classifier safeguards. That work matters, but the reports themselves show why a refusal cannot be the primary enterprise boundary. Attackers used several commercial models, open-weight models, stolen accounts, local gateways, and public offensive frameworks. Anthropic notes that a month-long hacktivist campaign ran entirely on stolen API keys. Google observed actors use tooling that could switch among Claude, Gemini, and Codex.

A refusal at one model endpoint may still leave the attacker with valid cloud credentials, an already-deployed agent framework, persistent campaign memory, or another provider. Conversely, a safety trigger can be abused defensively by malware to discourage an AI scanner from examining code. Organizations therefore need to collect refusal and abuse signals without assuming that the blocked model response stopped the surrounding process.

The useful detection point is the action boundary: a new service account, an OIDC token read from runner memory, a package publish, an unusual model-key use pattern, an agent writing startup hooks, a mass token validation job, or a sudden change in compute quotas. These events remain observable even when the model conversation is encrypted, distributed across providers, or absent because the attacker moved to a local model.

What defenders should change

  • Inventory the whole agent execution plane. Record model accounts, API keys, MCP servers, tool permissions, instruction files, memory stores, sandboxes, CI identities, cloud roles, and outbound destinations. A model inventory alone misses the path that performs actions.
  • Treat AI credentials as production credentials. Remove keys from repositories, mobile packages, container images, client-side code, and persistent agent memory. Use short-lived workload identity, audience restrictions, per-agent scopes, rapid revocation, and spend and geography anomaly detection.
  • Constrain feedback loops. Set hard limits on targets, concurrency, runtime, retries, network ranges, tool calls, and data volume. Require an independent policy gate before an agent can widen scope, create identity, publish a package, or act on newly harvested credentials.
  • Separate discovery from exploitation. A scanner should not automatically convert a hypothesis into a production exploit attempt. Bind authorization to the exact asset, technique, time window, and expected evidence; route ambiguous results to human review.
  • Monitor agent state as persistence. Alert on changes to hidden workspace directories, instruction files, hooks, tool manifests, MCP configuration, scheduled agent jobs, and long-lived memory. Rebuilding a host without cleaning synchronized project state can restore the attacker’s instructions.
  • Verify provenance beyond the badge. For signed packages, correlate the attestation with the expected workflow file, runner identity, source commit, reviewers, dependency graph, and build behavior. A valid signature from a stolen trusted-publisher token is still valid—and still malicious.
  • Harden CI token handling. Keep OIDC tokens short-lived and audience-bound, prevent untrusted code from sharing publisher jobs, isolate runners, restrict process inspection, pin actions, and make release authorization independent of the build process being attested.
  • Detect machine-speed transitions. Correlate secret discovery, token validation, privilege changes, cross-tenant enumeration, bulk export, and outbound transfer over minutes rather than reviewing each event in isolation. The reported intrusions moved from one credential to broad control in hours.
  • Use provider abuse signals as one sensor. Ingest account disruption, key-compromise, model-refusal, and impossible-travel signals where available, but enforce authorization, egress, and data controls outside the model and across providers.
  • Exercise autonomous attackers. Test defenses against agents that retry, change tools, rewrite payloads after detection, preserve memory, and exploit legitimate cloud or CI identity. A single blocked command does not end a closed-loop campaign.

The consequential change in these reports is not that models can write exploit code; defenders have known that trajectory for years. It is that attackers are assembling the surrounding machinery needed to keep working when a command fails, a signature fires, a target differs from the last one, or a credential opens an unexpected route. The defensive answer is the same architecture principle required for legitimate agents: keep authority narrow, make state and actions observable, and ensure no model can silently turn a successful observation into an expanded mandate.

Sources: