Is AI-Generated Code Secure? What the Studies Say

On October 29, Veracode published its fall report on AI-generated code: 18 models, four languages, four classes of vulnerability. The two best, Claude Opus 5.5 and GPT-6-Sol, produce secure code 73% of the time. Read it the other way round: the best model on the market leaves a known flaw in one task out of four.

Eight days earlier I had written about php2shell, the PHP flaw that an AI made demonstrable in a week. Put the two stories together and you get this year's paradox: the same technology writes vulnerable code and finds vulnerabilities that had escaped us for twenty years.

The question "is AI-generated code secure?" deserves an answer made of data, not opinions. I lined up the studies published from 2023 to today, keeping lab benchmarks separate from data collected in the field, and I looked at what the new tools do, starting with the security skill Cloudflare released as open source.

How much AI-generated code contains vulnerabilities

Veracode has measured this since 2025, always with the same method: it asks models to complete programming tasks in Python, Java, JavaScript and C#, where each task has a secure and an insecure way of writing the code, and it counts how often the result contains a known vulnerability. Four classes are measured: SQL injection, weak cryptography, cross-site scripting and log injection (CWE-89, 327, 80 and 117; CWEs are the public catalog of software weakness types).

The October 2026 numbers: the average rate of secure code rises from 56% to 62%, the share of vulnerable code drops from 44% to 38%, and ten models out of eighteen clear 60%, up from four in July. It's the first real improvement since the series began: from 2025 the average had sat at 55-56% while syntactic correctness got close to 100%.

The breakdown by vulnerability class cools the enthusiasm.

On SQL injection and cryptography the models have been good for a while, and there was little room left there. All of the improvement is in XSS, which started from 15% secure code. Log injection (writing user-supplied data to the logs without cleaning it) has been stuck at 12-13% in every snapshot: seven times out of eight, the model logs whatever came in with the request.

Two more figures from the report. Java is still the worst language, with 31% secure code against Python's 64%. And models specialized in code do worse than general-purpose ones, 51% against 54%: training a model to write more code doesn't train it to write secure code.

The other studies say the same thing with different methods

Secure Code Warrior had 16 models generate 1,760 complete applications, with a methodology developed together with RMIT University, and counted the confirmed vulnerabilities: around 15 per application on average, more than four of them severe. The most frequent weakness is CWE-532, sensitive information written to logs, with more than 8,500 occurrences. No single model came out as the most secure consistently.

A July 2026 study compared ChatGPT, Copilot and Gemini on automation scripts, the kind that ends up in business processes without anyone looking at it. Every script contained at least one exploitable vulnerability, and 9 of the 17 classes found showed up in all three models. The authors' conclusion: the risk depends on the type of task more than on the model.

One thing stands out if you work with PHP: none of these benchmarks measures it. Veracode covers four languages, Secure Code Warrior lists Java, Python, C# and C among its frameworks, and PHP doesn't appear. For Laravel and WordPress all we have is the analogy, and the analogy says the error classes are the same: unescaped output in views, hand-built queries, request data in the logs.

Why models keep failing in the same places

In its first report, in 2025, Veracode described the mechanism that explains everything else: every task allowed a secure and an insecure implementation, and in 45% of cases the models produced the insecure one. Not out of incompetence, since the syntax is perfect, but because the model reproduces the most frequent code in its training data, and the most frequent code on the internet is not the most secure. Tutorial examples log the whole request, concatenate strings into queries and skip validation for brevity.

The second cause is that the vulnerabilities in generated code are mostly omissions. The Cloud Security Alliance, in an April 2026 note on the security debt of vibe coding (building whole applications by describing them to a model, without reading the code it produces), puts missing access control, authorization and hardcoded credentials at the top of its categories. An omission is invisible: the code works, the tests pass, only the part that says who can do what is missing. The example the note reports is an AI chat platform with a Firebase database left open for reading: 406 million records belonging to around 25 million users exposed, in January 2026, because of an access rule that was never written.

The third cause is iteration. A study accepted at IEEE ISTAS 2025 took 400 code samples and had the model "improve" them repeatedly: after five rounds, critical vulnerabilities had increased by 37.6%. Security-oriented prompts help in the first iterations, then the improvements get offset by new, subtler flaws. It's the opposite of what happens when a programmer polishes their own code.

Then there's a risk that isn't about the code but about dependencies. A study presented at USENIX Security 2025 analyzed 576,000 code samples generated by 16 models: about one suggested package in five doesn't exist, for a total of 205,474 invented names, and 43% of those names come back identical on every request. That's enough for an attack technique with a name of its own, slopsquatting: you register the invented package with malicious code inside and wait for someone to run composer require or npm install without checking.

What it looks like in Laravel

Translated into Laravel, the three classes the benchmarks measure worst look like these lines. A model writes them gladly, because it has seen them thousands of times, and a review should stop them.

// CWE-532: the whole request in the logs, passwords and tokens included
Log::info('Login attempt', $request->all());

// CWE-89: query built by concatenating user input
$users = DB::select("SELECT * FROM users WHERE email = '" . $request->email . "'");

// CWE-80: output printed in a Blade view without escaping
{!! $comment->body !!}

In plain PHP they're the same three lines with error_log and $_POST, a concatenated query on PDO and an echo without htmlspecialchars. None of the three fails a functional test. All three get through a review done in a hurry.

The problem isn't just the model: it's whoever uses it

The first serious study on this topic dates from 2023 and doesn't measure the model, it measures people. At Stanford, Perry, Srivastava, Kumar and Boneh put 47 programmers in front of five security tasks, two thirds with a Codex-based assistant and one third without. On the Python encryption task, 43% of those working alone wrote a secure solution, against 21% of those with the assistant. On the SQL query in JavaScript, injection-vulnerable code went from 7% to 36%.

The figure that matters most is a different one: those with the assistant were more convinced they had written secure code, and those who trusted the assistant more wrote the least secure code. It's the same mechanism I discussed about hallucinations and people who never verify: trust takes the place of checking.

The 2023 model was codex-davinci-002, a long way from today's. But the data collected in the field in 2025 and 2026 says human behavior hasn't changed. Apiiro analyzed repositories from developers at Fortune 50 companies: those using assistants produce three to four times more code, expose keys and credentials almost twice as often, and pack more changes into fewer pull requests, which become harder to read.

Security findings in their sample grew tenfold between December 2024 and June 2025. Take the figure with caution, because it comes from a vendor of security tools and the velocity part has been disputed; the direction, though, matches everything else.

Finally the CVEs, the public flaws with an official identifier. Georgia Tech's Vibe Security Radar reconstructs, through the fixing commits, how many published vulnerabilities come from generated code: 6 in January 2026, 15 in February, 35 in March, 74 in total by the end of March. The researchers estimate the unattributed ones at five to ten times that, because when the commit leaves no trace of the tool used, the flaw stays anonymous.

If you lead a team, these figures read like this: the tool hasn't lowered the quality of the individual programmer, it has raised the volume of code arriving for review and lowered the amount of attention per line. The bottleneck has moved from whoever writes to whoever reads, that is, to code review.

The AI that finds the flaws: Cloudflare, Anthropic and OpenAI

The flip side is the other half of this year's news. php2shell was demonstrated by one researcher with a model in a week, work that used to take a team months. In May I wrote about Project Glasswing, Anthropic's decision to give Claude Mythos to only about forty organizations because it was too good at finding vulnerabilities. Six months later, the same models that write 38% vulnerable code have become security reviewers, and no longer just for a few.

In June Cloudflare explained how it built its own vulnerability research system and released, under the MIT license, the skill it all started from: security-audit, a file of instructions with lists of attack classes and small validators, which a coding agent (Claude Code, Codex and the like) loads when you ask it for an audit. The work is split into six phases: architecture reconnaissance, hunting by attack class with isolated agents, validation of every candidate, structured output, independent verification and reporting.

The idea that holds it all together is one sentence from the post:

If a Hunter is allowed to grade its own homework, it will confidently validate everything it outputs.

That's why every candidate goes to a fresh verifier, one that took no part in the hunt and whose only job is to tear it apart. A finding is "confirmed" only with a complete trace in the source and an observed effect; otherwise it stays "needs validation", with no severity. Across its own fleet of 128 repositories, Cloudflare went from 20,799 raw candidates to 7,245 findings handed to engineering teams, and the rejection rate in validation fell from 40% to 11% as the context passed to the verifiers improved. A single run finds about half of the flaws that repeated runs find: the public skill is the seed, the full system hasn't been released yet.

Two limits the documentation states. Without an OS-enforced sandbox (no network, memory limits, confined writes) the skill doesn't execute the code under analysis, and conclusions that depend on runtime behavior stay "needs validation". And the output is reports, not patches: the fix is left to whoever reads them.

I've seen it at work on a mid-sized Laravel application, with the quick profile: half an hour and 19 agents, four for reconnaissance, four hunters, one critic and ten verifiers, one per candidate. The result: one confirmed finding, eight leads needing validation and one rejected with the reason written down.

The leads weren't vague. Each came with the trace in the source, the fact the code alone can't prove (the value of an environment variable in production, how the web server normalizes a URL) and a test already written to close it in a minute. On macOS the sandbox is missing, and everything that depended on runtime behavior ended up in that column. That's the right behavior: a reviewer who doesn't know says it doesn't know.

Anthropic and OpenAI made the same choice with closed products. Claude Security has been in public beta since May 2026, and Anthropic claims the previous model had already found more than 500 high-severity vulnerabilities in open-source code that had passed under experts' eyes for decades; in July came the plugin for Claude Code, which scans locally and proposes patches.

Codex Security, formerly Aardvark, has been in preview since March, and OpenAI reports false positives falling by more than 50% from one run to the next. Google's Big Sleep had already published 20 vulnerabilities in open-source projects in the summer of 2025.

The common thread isn't the model, it's the scaffolding around it: one agent proposes, another contests, a person decides. Which is exactly what's missing when a model writes code and nobody reads it.

What changes for whoever writes and reviews code

I take the practical conclusions from the reports, which agree on this.

Generated code is an unreviewed external contribution. Veracode says so in plain words, and the CSA turns it into rules: static analysis (SAST, the tools that read source code hunting for dangerous patterns) and secret scanning as a mandatory step before merge, not a spot check.

Review should be allocated by risk, not by lines. Authentication, authorization, cryptography and data access are where the models omit things, and they're the areas that need a reviewer who knows what to look for. A Blade view and a Laravel policy don't deserve the same attention.

Pull requests should stay small. Apiiro's finding about bigger, rarer PRs is the one a team can fix on its own, with no new tools.

Suggested dependencies need checking. A package proposed by a model should be looked up on Packagist or npm before installing it, and a package created recently with few downloads deserves a second look.

Blind iteration makes things worse. "Make it faster" repeated five times produces more vulnerable code, the iteration study says so. Every round needs reading.

Automated auditors are a second pair of eyes, not the first. Cloudflare's skill and the Anthropic and OpenAI products cost tokens, not licenses, and they catch things a human review lets through. But they work by verifying candidates, and a candidate nobody has read is still a candidate.

If you work in PHP, with or without Laravel, the extra advice is not to wait for a benchmark that may never come: the three lines in the example above are the minimum to look for in every review.

Where this is going

The honest answer to the question in the title is that AI-generated code is about as secure as code written in a hurry by someone who trusts themselves, which is to say not very, with two differences. The first is volume: three to four times as much of it arrives. The second is confidence: the Stanford study says those who use it believe it's more secure than it is.

The numbers are improving, but on the low-hanging fruit. Veracode's 62% came from fixing XSS, while log injection has been stuck since 2025 and authorization omissions don't show up in any benchmark. I don't believe a model at 73% becomes one at 95% in 2027, because the limit isn't the model's capability: it's that it reproduces the code we fed it.

What really changes is where the bottleneck sits. Writing code is cheap, reading it costs what it always did, and finding flaws keeps getting cheaper for defenders and attackers alike, as php2shell showed. Who, on your team, reads the code the model wrote? If the answer is "nobody, the tests pass", 38% is your number.