Make money doing the work you believe in

The weapon had footnotes. It came with a study guide, a severity score the AI invented, and comments explaining how each step of the attack worked. The exploit was formatted like a textbook. It read like a tutorial. And it was designed to bypass two-factor authentication on a popular web administration tool used by organizations worldwide, as the opening move of a planned mass exploitation campaign.

On May 11, Google’s Threat Intelligence Group published the first confirmed case of a zero-day exploit developed with artificial intelligence and deployed by criminal threat actors in the real world. GTIG assessed with high confidence that an AI model was used to both discover and weaponize a semantic logic flaw, a hardcoded trust assumption in a 2FA authentication flow, then generate a working Python script to exploit it. Google worked with the unnamed vendor to patch the vulnerability and believes its intervention disrupted the campaign before it gained traction.

The AI signatures in the code are what make this structurally different from every previous exploit. The script contained educational docstrings, inline comments that explain the attack’s own logic step by step in the style of teaching material. It included a hallucinated CVSS severity score, a rating the AI generated from its training data rather than from any real vulnerability database. It used structured, textbook Python formatting with detailed help menus and clean class definitions characteristic of large language model output. No experienced human attacker writes exploit code with pedagogical annotations. The AI wrote the weapon the way it would write a lesson.

The vulnerability itself reveals why this changes the offense-defense balance permanently. Traditional security scanners detect buffer overflows, memory corruption, and known vulnerability patterns. They do not read code the way a developer writes it. Large language models do. The flaw was a semantic logic error, a contradiction between the developer’s intent and the code’s actual behavior, buried in a trust assumption that looked functionally correct to every automated tool in existence. The AI correlated intent with implementation and found where they diverged. That is a category of vulnerability discovery that traditional tooling is structurally unable to perform.

GTIG chief analyst John Hultquist framed the implications directly: “For every zero-day we can trace back to AI, there are probably many more out there.” The visible exploit is the surface. The undetected ones are the substrate.

The same report documents the broader landscape. North Korean group APT45 has been sending thousands of repetitive prompts to AI models to recursively analyze vulnerabilities and build an exploit arsenal at a scale impractical without automation. A China-linked actor used expert-persona jailbreaks to push Gemini into researching pre-authentication remote code execution flaws in router firmware. Russian operations are splicing AI-generated audio into legitimate news footage. An Android backdoor called PROMPTSPY uses Gemini API calls to autonomously navigate infected devices, capture biometric data, and replay authentication gestures. And in March, criminal group TeamPCP compromised LiteLLM, a widely used AI gateway library, by embedding a credential stealer through poisoned PyPI packages.

The AI that identifies your groceries and the AI that bypasses your authentication run on the same foundational architecture. Google published the GTIG report on May 11. Samsung began deploying Google’s Gemini into refrigerators the same day. The dual use is no longer theoretical. It shipped in the same news cycle.

May 13
at
12:51 AM
Relevant people

Log in or sign up

Join the most interesting and insightful discussions.