Blitzy Blows Past SWE-bench Verified, Demonstrating Next Frontier in AI Progress

New autonomous code generation platform achieves breakthrough performance through extended inference time compute, signaling new paradigm beyond pretraining scaling

CAMBRIDGE, Mass., Sept. 9, 2025 /PRNewswire/ -- Blitzy, the autonomous software engineering orchestration platform, today announced it has achieved the top position on SWE-bench Verified, the industry's leading benchmark for AI coding capabilities. Blitzy's 86.8% performance, represents a 13.02% improvement (10 percentage point leap) over the previous best — the largest single advance since March 2024 when Devin achieved a 6.9% improvement (11.9 percentage point leap) over existing state-of-the-art models. The result demonstrates Blitzy's technical excellence and that inference time scaling delivers exponential rather than incremental improvements, and establishes the company's credibility as the leader in autonomous software development.

Blitzy's unprecedented result comes at a time when AI progress has notably decelerated across multiple dimensions. Pre-training improvements have become increasingly incremental compared to the dramatic jumps of previous generations. Even single models appear to be hitting performance ceilings, with leading systems clustering around 70-75% on SWE-bench Verified (a plateau suggesting fundamental limitations in current approaches). While reasoning capabilities have shown promise through foundation models including OpenAI's recent GPT-5 release, the true potential of scaling inference time compute to drive exponential results remained largely unproven — until now.

Smashing Through The "Unsolvables" Ceiling

Blitzy's 86.8% performance isn't just a benchmark victory — it's a breakthrough beyond what the AI community considered the practical ceiling for SWE-bench Verified. OpenAI's analysis during the creation of SWE-bench Verified found that human evaluators identified numerous samples as "hard or impossible to solve," due to ambiguous issue descriptions, insufficient context, or contradictory requirements that rapid AI systems couldn't navigate. Previous attempts plateaued as they encountered these "unsolvable" problems that stumped single-pass reasoning.

"The 'unsolvables' weren't actually unsolvable — they just required deeper thinking than System-1 AI could provide," explained Sid Pardeshi, Blitzy CTO and Co-founder. "By design, our platform enables AI to think for hours or days rather than seconds or minutes, unlocking solutions to problems that stumped every previous approach. This validates inference time scaling as the key to exponential capability improvements."

Blitzy's System-2 approach transforms these roadblocks into solvable challenges through extended reasoning time. This accomplishment signals that the ceiling for complex problem-solving isn't determined by problem difficulty, but by reasoning depth. As Blitzy demonstrates, with sufficient thinking time, AI systems can break through barriers that seemed insurmountable under time pressure.

Blitzy's SWE-bench Verified performance may signal a fundamental shift in how companies develop AI coding solutions. The industry's current emphasis on quick responses and immediate feedback has started to give way to more in-depth reasoning and the higher quality solutions that follow.

The Evolution of Benchmarks

The Foundation Era

This evolution is reflected in the benchmark landscape itself. SWE-bench Verified served an important purpose when AI coding capabilities were nascent, providing standardized evaluation for models attempting basic programming tasks. The benchmark proved AI could move beyond code completion to actual problem-solving, establishing credibility for the entire autonomous coding category.

For over a year, SWE-bench Verified remained the gold standard, driving incremental progress that pushed performance from 13.86% (March 2024) to previous leaders reaching 76.8%. This incremental progression served the industry well, providing clear metrics for comparing approaches and validating improvements.

However, recent research has highlighted evaluation challenges inherent in any static benchmark. Studies indicate that 32.67% of SWE-bench's patches may involve solution leakage — where problem descriptions inadvertently contain guidance — while 94% of issues predate LLM training data, raising questions about whether high performance reflects genuine reasoning or pattern recognition from training. These findings illuminate the complexity of measuring true AI capabilities versus optimized performance on known problem sets.

The Next AI Frontier: System-2 Everywhere

As AI capabilities have rapidly matured, the benchmark's limitations became apparent. The focus on isolated Python bug fixes disconnects from enterprises' realities that require sustained reasoning across massive codebases, architectural transformation, and multi-step workflow orchestration. Research from Berkeley AI published in February 2024 predicted that cutting-edge results would increasingly emerge from compound AI systems rather than individual models. Blitzy's SWE-bench performance validates this prediction and has proven it works at both benchmark and enterprise scale.

Inference time compute is the scaling frontier that enables exponential rather than incremental AI progress. Unlike pretraining scaling with its resource constraints and diminishing returns, inference time scaling offers unlimited improvement potential bounded only by problem complexity and computational budget allocation.

This paradigm extends far beyond coding. Medical diagnosis, financial analysis, legal research, and engineering design all represent domains requiring careful consideration and multi-step reasoning that benefit from extended inference time approaches. The transition from System-1 to System-2 AI represents the next exponential improvement curve the industry has been seeking.

Enterprise Validation: Beyond Benchmarks

Blitzy's 86.8% SWE-bench Verified performance validates its technical excellence, but its enterprise impact reveals capabilities that current benchmarks fundamentally cannot measure. Its real-world transformations demonstrate the exponential power of inference time compute — proving AI can architect, modernize, and transform entire systems at unprecedented scale. This enterprise-scale context management and multi-step workflow orchestration represents the next frontier beyond isolated coding benchmarks.

Examples:

• Modernization of 4 million lines of legacy Java leveraging 72+ hours of distributed reasoning time per major architectural decision — complexity impossible with time-constrained approaches.
   
• Service extraction from 500,000-line monoliths that requires 24+ hours of architectural analysis to identify optimal boundaries and integration patterns.
   
• Cross-language migration while maintaining mathematical precision through extended verification cycles which ensures semantic equivalence across algorithmic transformations.

About Blitzy

Blitzy is the System-2 AI code generation platform that achieved breakthrough performance on SWE-Bench Verified through extended inference time compute and multi-agent orchestration. Unlike traditional AI coding tools that rely on rapid single-pass generation, Blitzy enables hours or days of reasoning time for complex enterprise challenges, coordinating multiple specialized agents to deliver comprehensive solutions.

The platform maintains coherent understanding across multi-million-line codebases, enabling semantic-preserving transformation between programming languages and entire technology stacks. Blitzy orchestrates comprehensive redevelopment rather than incremental patches, coordinating complex development processes from requirements through deployment while engaging in progressive refinement cycles that optimize results far beyond single-pass generation.

Enterprise customers across financial services, professional services, and technology sectors rely on Blitzy's extended reasoning capabilities to solve problems that require architectural depth rather than coding speed — transforming legacy systems, extracting services from monolithic applications, and modernizing entire technology ecosystems through sustained AI reasoning that no benchmark currently measures.

Media Contact(s): Brian Elliott, [email protected]

Source(s):

SWE-Bench http://swebench.com

White paper:

https://paper.blitzy.com/blitzy_system_2_ai_platform_topping_swe_bench_verified.pdf

SOURCE Blitzy