New Data Shows AI Is Changing What Software Engineers Do and the Skills They Need Most

From Coders to AI Quality Guardians, Engineers are Shifting Focus — with 66% of Leaders Saying that Validating AI Outputs is Now a Critical Skill, According to Uplevel Survey

SEATTLE, Sept. 9, 2025 /PRNewswire/ -- AI is transforming software engineering at record speed — reshaping what engineers do and the skills they need to succeed. As AI adoption increases, a new survey from Uplevel shows what engineering leaders believe will be the most important skills for their teams:

• Validation of AI outputs and quality assurance (QA) — cited by two-thirds of leaders (66%)
• Performance monitoring and optimization — 39%
• System architecture and integration skills — 34%

These findings come from Uplevel's new report, The AI Measurement Crisis, with data from 100+ senior engineering leaders at mid-to-large technology companies. Conducted online in June 2025, the study explored leaders' views on AI risks and opportunities, along with broader factors shaping engineering productivity.

Coding Stands to Change the Most

With AI increasingly embedded in engineering workflows, half of leaders (50%) say code generation is the activity most likely to require less human effort, as well as the area most likely to be transformed by AI (56%).

That change is already underway. Microsoft recently shared that its AI coding tool, GitHub Copilot, has reached 20 million all-time users, with deployment by 90% of the Fortune 100. But with speed comes tradeoffs: A previous Uplevel analysis found a 41% increase in bug rates with generative AI (GenAI) for coding — underscoring why QA is now seen as mission-critical.

"The potential of AI to deliver customer value is far greater than just code generation," said Uplevel CEO Joe Levy. "Leaders should consider new use cases that clarify customer needs and automate time-consuming tasks like reviews, deployments and testing. That's where AI begins to deliver real customer value — in the outcomes, not just the code."

Racing Toward AI…  and Straight Into a Technical Debt Traffic Jam?

Across software engineering, it's pedal to the metal for AI adoption, amidst mounting executive pressure — with CEOs across industries expecting to double AI investment growth rates in the next two years, often impelled by "the risk of falling behind."

Rushing isn't without risks, though. According to Uplevel's study, nearly 9 in 10 engineering leaders (87%) say their business is "prepared" or "very prepared" to implement AI solutions — yet they also worry that hidden bottlenecks could slow AI's long-term impact.

Top of the list? Technical debt — the extra work created by quick software fixes that speed things up now… but create headaches and complexity later. More than one in four engineering leaders (27%) see it as the greatest strategic threat to AI's potential, followed by a lack of clear AI strategy (22%).

These concerns come when technical debt is already having a tangible impact:

• A quarter of engineering leaders (25%) say technical debt is the single biggest constraint on their team's ability to deliver value.
• 21% report it slows delivery speed more than any other factor.
• Nearly 1 in 5 (19%) believe reducing technical debt would yield the biggest productivity gains.

"The promise of AI is speed — but code generation itself is not the bottleneck," said Amy Carrillo Cotten, director of client transformation at Uplevel. "Our data shows that technical debt — more than any other factor — blocks engineering teams from delivering value. Addressing that now ensures you're not just going faster, but delivering value that actually lasts."

Technical debt isn't the only issue on engineering leaders' minds. When asked separately about their most urgent concerns for AI implementations, engineering leaders cite:

• Data security and privacy risks — Nearly 1 in 3 (30%)
• Quality control and reliability issues — Approximately 1 in 5 (19%)
• Skills gap, with a lack of AI expertise — Approximately 1 in 5 (18%)

To equip their teams for an AI-driven future, engineering leaders say their businesses are taking various approaches:

• Reskilling existing employees — 40%
• Hiring new AI specialists — 34%
• Partnering with vendors or consultants — 22%

The AI Measurement Crisis

Equipping teams with new AI skills is only part of the equation. Whether those investments pay off depends on how organizations measure success — and here, many are still using outdated playbooks.

Engineering leaders want AI to:

• Increase operational efficiency — 53%
• Accelerate innovation — 40%
• Improve decision-making — 28%
• Boost competitive advantage — 23%

Yet their measurement habits lag behind their ambitions. Many still lean on individual productivity metrics, even though their biggest delivery constraints are systemic — including cross-team dependencies (31%), complex architectures and technical debt (21%), and unclear project requirements (14%).

When it comes to GenAI, the pattern holds. The top two metrics leaders use to measure effectiveness are developer productivity and reduction in error rates — with less focus on broader, business-critical outcomes like cost savings, speed-to-market and customer satisfaction.

And while two-thirds (66%) of engineering leaders say they regularly measure the business outcomes of their teams' work, barriers like difficulty isolating team performance (37%) and a lack of the right tools (21%) keep organizations stuck optimizing what's easy to count — not what actually drives impact.

"Until leaders modernize their measurement frameworks, the very outcomes they hope AI will deliver may remain stubbornly out of reach," Uplevel CEO Joe Levy said. "The organizations that get it right will look beyond activity metrics — tracking how AI improves teamwork, accelerates delivery, and drives business results that matter."

For more information and insights from Uplevel's The AI Measurement Crisis Report, please see https://resources.uplevelteam.com/ai-measurement.

About Uplevel
Uplevel is the engineering optimization system that makes it easier for tech leaders to deliver value sustainably and on time. Applying advanced data science to tooling and collaboration data, Uplevel surfaces the hard-to-find signals and enables the changes that organizations need to focus their efforts, prioritize initiatives, and build an effective engineering culture. With Uplevel, software development organizations worldwide push the limits of product velocity and generate exceptional ROI — making reliable decisions with data. For more, visit uplevelteam.com.

SOURCE Uplevel