Atomicwork and New Measure Unveil ITSMBench, a New Standard for Evaluating Frontier AI in IT Service Management

Open-source benchmark shows that even the best-performing model completes only 50.56% of requests typically encountered by Enterprise IT service teams.

SAN FRANCISCO, Aug. 12, 2026 /PRNewswire/ -- Atomicwork, the enterprise ITSM and ESM platform with an AI workforce to run it, and New Measure today announced ITSMBench, the first benchmark designed to evaluate how effectively models from OpenAI, Anthropic, xAI, GLM, and other developers and agent frameworks perform real-world IT Service Management work. By recreating the systems, workflows, and ambiguity that define modern enterprise service desks, ITSMBench gives the industry a realistic way to measure what AI agents can actually accomplish today.

"Frontier models are brilliant at writing code, but they are completely blind to the hidden security landmines inside enterprise workflows," said Vijay Rayapati, co-founder and CEO of Atomicwork. "ITSMBench proves that blindly trusting an AI agent right now is an invitation for an enterprise security breach."

A benchmark built around real enterprise IT work

Built jointly by Atomicwork and New Measure, ITSMBench recreates enterprise environments using 42 mocked software systems backed by nearly 1,800 database tables and more than 2,000 REST endpoints. The benchmark evaluates 89 service desk tasks spanning the kinds of L2 and L3 investigations enterprises handle every day, including identity and access management, devices, networking, security operations, infrastructure, and engineering escalations. Every environment, task, and evaluation is open source, allowing researchers and enterprises to reproduce results and build on the benchmark.

No frontier model comes close to completing every task

Initial evaluations show that today's frontier models exhibit distinct strengths and weaknesses with different harnesses:

• Grok 4.5 leads at tool discovery, finding the right APIs 83.0% of the time, but ranks near the bottom on actually finishing the job.
• Opus 5 performs best on execution, completing 63.5% of tasks once the right tools are in hand, despite having the weakest discovery rate at 70.2%.
• GPT-5.6 Sol splits the difference, landing between Grok 4.5 and Opus 5 across discovery and execution.
• GLM-5.2 is far cheaper, costing $0.25 per trial, but trails both Opus 5 and GPT-5.6 Sol on execution.

The benchmark also demonstrates that agent frameworks can materially affect both accuracy and cost, and that models frequently stop investigating after finding a plausible explanation instead of verifying the true root cause. Even when agents identify the correct resolution, they often leave related systems, records, or follow-up actions incomplete. Across the benchmark, 73 of 89 tasks were successfully completed at least once by at least one model, although no single model solved the full set.

"Enterprise service management demands more than reasoning," said Arushi Gandhi, CEO at New Measure. "One model fits all is seldom the right answer. Enterprises that want an AI-run service desk need to optimize for accuracy, cost, and time to resolution at the same time. Getting this right, thus, takes a constellation of orchestrated models. ITSMBench is our proposed framework for CIOs and IT leaders to evaluate which models/harnesses are best for them for their enterprise IT needs."

The ITSMBench benchmark, methodology, benchmark results, and open-source environments are available at atomicwork.com/itsm-bench.

About Atomicwork

Atomicwork is an AI-native ITSM and ESM platform that comes with an AI workforce to run it — purpose-built for companies scaling through intelligence, not just headcount. Its governed AI Coworkers run access management, incident response, onboarding, and other enterprise service delivery initiatives end-to-end. From AI-native scaleups to global enterprises, organizations of all shapes and sizes run on Atomicwork. The company is headquartered in Palo Alto, CA, and backed by leading investors including Khosla Ventures, Battery Ventures, Blume Ventures and Z47.

SOURCE Atomicwork Inc