Artificial Intelligence & Machine Learning
,
Next-Generation Technologies & Secure Development
Machine-Written Pull Requests Contain 70% More Bugs

Artificial intelligence coding assistants write code faster than humans. They also write buggier code, though nobody puts that in the marketing materials.
See Also: A CISO’s Perspective on Scaling GenAI Securely
Researchers at code review tool CodeRabbit analyzed 470 open-source pull requests on GitHub, analyzing AI-coauthored submissions against human-only contributions for their logic, maintainability, security and performance.
AI-generated pull requests averaged nearly 11 issues each, compared with roughly six in human-generated submissions, resulting in longer review cycles and higher risk of defects reaching production.
Beyond the volume, the severity of the bugs also escalated with AI involvement. CodeRabbit classifies issues as “critical” when they could cause system failures, security breaches or data loss, and “major” when they significantly impact functionality or performance. Both categories appeared substantially more often in machine-written code.
AI-generated code showed significantly higher rates across multiple vulnerability categories. Improper password handling increased by 88%, insecure object references climbed by 91%, cross-site scripting vulnerabilities nearly tripled and insecure deserialization implementations rose 82% compared to human developers.
David Loker, director of AI at CodeRabbit, connected the findings to broader industry observations. “These findings reinforce what many engineering teams have sensed throughout 2025,” Loker said. “AI coding tools dramatically increase output, but they also introduce predictable, measurable weaknesses that organizations must actively mitigate.”
The use of AI code generation has expanded across the industry, with organizations positing gains such as faster engineering speed and reduced time spent on repetitive work, though the quality implications are now becoming clearer. One developer chronicling his use of AI tools to build a networking application earlier this year tweeted that the AI went “rogue during a code freeze” and deleted the entire database. A randomized controlled trial published in July found that developers using AI tools believed they increased their speed by 20% – but actually slowed down by 19%.
The study identified two areas where AI outperformed human developers: Machines made fewer spelling errors and produced code with fewer testability issues.
The research drew exclusively from open-source GitHub repositories, a methodology CodeRabbit acknowledged carries limitations. The company said it could not verify with certainty that pull requests labeled as human-authored contained no AI assistance. The 320 AI-coauthored submissions carried explicit AI labeling, while the 150 human-only entries lacked such markers.
The findings also somewhat contrast with some earlier academic research on AI code generation. An August paper from University of Naples researchers said that AI-generated Python and Java code “is generally simpler and more repetitive, yet more prone to unused constructs and hardcoded debugging, while human-written code exhibits greater structural complexity and a higher concentration of maintainability issues.”
Researchers from Monash University in Australia and the University of Otago in New Zealand published findings in January showing that GPT-4 frequently produces more complex code, potentially requiring additional reworking for maintainability, though a higher number of test cases passed for GPT-4 generated code across tasks compared to human-written submissions.
Microsoft has patched more than 1,100 CVEs this year, which according to Trend Micro researcher Dustin Childs, is the second-largest year for CVEs by volume since 2020. Microsoft CEO Satya Nadella has earlier said that up to 40% in certain repositories originated from AI, and Copilot Actions separately includes cautionary language about “the security implications of enabling an agent on your computer.” Childs wrote that as Microsoft’s portfolio expands and AI bugs become more prevalent, CVE numbers will likely climb higher in the coming year.
