Accessibility Statement Skip Navigation
  • Resources
  • Data Privacy
  • Contact Us
  • Send a Release
Return to PR Newswire homepage
  • News
  • Products
  • Contact
Search
When typing in this field, a list of search results will appear and be automatically updated as you type.

Searching for your content...

No results found. Please change your search terms and try again.
  • News Releases view all

    • News in Focus
        • Browse News Releases

        • All News Releases
        • All Public Company
        • All Multimedia

        • All Multimedia
        • All Photos
        • All Videos
    • Business & Money
        • Auto & Transportation

        • Aerospace & Defense
        • Air Freight
        • Airlines & Aviation
        • Automotive
        • Maritime & Shipbuilding
        • Railroads & Intermodal Transportation
        • Supply Chain/Logistics
        • Transportation, Trucking & Railroad
        • Travel
        • Trucking & Road Transportation
        • View All Auto & Transportation

        • Business Technology

        • Blockchain
        • Broadcast Tech
        • Computer & Electronics
        • Computer Accessories
        • Computer Hardware
        • Computer Networks
        • Computer Software
        • Data Analytics
        • Electronic Commerce
        • Electronic Components
        • Electronic Design Automation
        • Financial Technology
        • High-Tech Security
        • Internet Technology
        • Nanotechnology
        • Semiconductors
        • View All Business Technology

        • Entertain­ment & Media

        • Advertising
        • Art, Culture & Design
        • Books
        • Entertainment
        • Film & Motion Picture
        • Magazines
        • Music
        • Publishing & Information Services
        • Radio & Podcast
        • Television
        • View All Entertain­ment & Media

        • Financial Services & Investing

        • Accounting News & Issues
        • Acquisitions, Mergers & Takeovers
        • Banking & Financial Services
        • Bankruptcy
        • Bond & Stock Ratings
        • Conference Call Announcements
        • Contracts
        • Cryptocurrency
        • Dividends
        • Earnings
        • Earnings Projections or Forecasts
        • Financing Agreements
        • Insurance
        • Joint Ventures
        • Mutual Funds
        • Offerings
        • Private Placement
        • Real Estate
        • Restructuring & Recapitalization
        • Sales Reports
        • Shareholder Activism
        • Shareholder Meetings
        • Venture Capital
        • View All Financial Services & Investing

        • General Business

        • Awards
        • Commercial Real Estate
        • Corporate Expansion
        • Earnings
        • Environmental, Social and Governance (ESG)
        • Human Resource & Workforce Management
        • Licensing/marketing agreements
        • New Products & Services
        • Obituary
        • Outsourcing Businesses
        • Overseas Real Estate (Non-US)
        • Personnel Announcements
        • Residential Real Estate
        • Small-Business Services
        • Socially Responsible Investing
        • Surveys, Polls & Research
        • Trade Show News
        • View All General Business

    • Science & Tech
        • Consumer Technology

        • Artificial Intelligence
        • Blockchain
        • Cloud Computing/Internet of Things
        • Computer Accessories
        • Computer Electronics
        • Computer Hardware
        • Computer Networks
        • Computer Software
        • Consumer Electronics
        • Cryptocurrency
        • Data Analytics
        • Electronic Commerce
        • Electronic Design Automation
        • Financial Technology
        • Mobile Devices/Apps
        • Social Media
        • STEM (Science, Tech, Engineering, Math)
        • Wireless Communications
        • View All Consumer Technology

        • Energy & Natural Resources

        • Alternative Energies
        • Chemical
        • Electrical Utilities
        • General Manufacturing
        • Mining
        • Mining & Metals
        • Natural Gas Utilities
        • Oil & Energy
        • Oil & Gas Discoveries
        • Utilities
        • Water Utilities
        • View All Energy & Natural Resources

        • Environ­ment

        • Conservation & Recycling
        • Environmental Issues
        • Environmental Policy
        • Environmental Products & Services
        • Green Technology
        • Natural Disasters
        • View All Environ­ment

        • Heavy Industry & Manufacturing

        • Aerospace & Defense
        • Agriculture
        • Chemical
        • Construction & Building
        • General Manufacturing
        • HVAC (Heating, Ventilation & Air-Conditioning)
        • Machinery
        • Machine Tools, Metalworking & Metallurgy
        • Mining
        • Mining & Metals
        • Paper, Forest Products & Containers
        • Precious Metals
        • Textiles
        • Tobacco
        • View All Heavy Industry & Manufacturing

        • Telecomm­unications

        • Computer Accessories
        • Computer Networks
        • Mobile Devices/Apps
        • Telecommunications
        • Telecommunications Carriers & Services
        • Telecommunications Equipment
        • VoIP (Voice over Internet Protocol)
        • Wireless Communications
        • View All Telecomm­unications

    • Lifestyle & Health
        • Consumer Products & Retail

        • Animals & Pets
        • Beers, Wine & Spirits
        • Beverages
        • Cannabis
        • Cosmetics and Personal Care
        • Fashion
        • Food
        • Furniture & Furnishings
        • Home Improvement
        • Household, Consumer & Cosmetics
        • Household Products
        • Jewelry
        • Non-Alcoholic Beverages
        • Office Products
        • Product Recalls
        • Restaurants
        • Retail
        • Supermarkets
        • Toys
        • View All Consumer Products & Retail

        • Entertain­ment & Media

        • Advertising
        • Art, Culture & Design
        • Books
        • Entertainment
        • Film & Motion Picture
        • Magazines
        • Music
        • Publishing & Information Services
        • Radio & Podcast
        • Television
        • View All Entertain­ment & Media

        • Health

        • Biometrics
        • Biotechnology
        • Clinical Trials & Medical Discoveries
        • Dentistry
        • FDA Approval
        • Fitness/Wellness
        • Health Care & Hospitals
        • Health Insurance
        • Infectious Disease Control
        • International Medical Approval
        • Medical Equipment
        • Medical Pharmaceuticals
        • Mental Health
        • Pharmaceuticals
        • Supplementary Medicine
        • View All Health

        • Sports

        • Outdoors, Camping & Hiking
        • Sporting Events
        • Sports
        • Sports Equipment & Accessories
        • View All Sports

        • Travel

        • Amusement Parks & Tourist Attractions
        • Gambling & Casinos
        • Hotels & Resorts
        • Outdoors, Camping & Hiking
        • Passenger Aviation
        • Travel
        • View All Travel

    • Policy & Public Interest
        • Policy & Public Interest

        • Animal Welfare
        • Corporate Social Responsibility
        • Economic News, Trends & Analysis
        • Education
        • Environmental Products & Services
        • European Government
        • Natural Disasters
        • Not-for-Profit
        • Public Safety
        • View All Policy & Public Interest

    • People & Culture
        • People & Culture

        • Children-related news
        • Disabled Persons
        • Diversity, Equity & Inclusion
        • Hispanic-oriented news
        • LGBTQ+
        • Religion
        • Senior Citizens
        • Veterans
        • Women-Related news
        • View All People & Culture

    • Explore Our Platform
    • Plan Campaigns
    • Create with AI
    • Distribute Press Releases
    • Report Results
    • Amplify Content
    • All Products
    • General Inquiries
    • Request a Demo
    • Partnerships
    • Media Inquiries
    • Hamburger menu
    • PR Newswire: news distribution, targeting and monitoring Home
    • Send a Release
      • ALL CONTACT INFO
      • Contact Us


    • Searching for your content...

      No results found. Please change your search terms and try again.

      News Releases

      view all
      Close menu
    • News
    • Products
    • Contact
      • News Releases
      • Send a Release
      • Data Privacy
      • 2 News in Focus
      • 5 Business & Money
      • 5 Science & Tech
      • 5 Lifestyle & Health
      • 1 Policy & Public Interest
      • 1 People & Culture
      • News Releases
      • Send a Release
      • Data Privacy
      • Explore Our Platform
      • Plan Campaigns
      • Create with AI
      • Distribute Press Releases
      • Report Results
      • Amplify Content
      • All Products
      • News Releases
      • Send a Release
      • Data Privacy
      • General Inquiries
      • Request a Demo
      • Partnerships
      • Media Inquiries
      • Editorial Bureaus
      • News Releases
      • Send a Release
      • Data Privacy
      Close submenu (News in Focus) News in Focus
      • Browse News Releases
      • All Multimedia
      Close submenu (Business & Money) Business & Money
      • Auto & Transportation
      • Business Technology
      • Entertain­ment & Media
      • Financial Services & Investing
      • General Business
      Close submenu (Science & Tech) Science & Tech
      • Consumer Technology
      • Energy & Natural Resources
      • Environ­ment
      • Heavy Industry & Manufacturing
      • Telecomm­unications
      Close submenu (Lifestyle & Health) Lifestyle & Health
      • Consumer Products & Retail
      • Entertain­ment & Media
      • Health
      • Sports
      • Travel
      Close submenu (Policy & Public Interest) Policy & Public Interest
      • Policy & Public Interest
      Close submenu (People & Culture) People & Culture
      • People & Culture

      New Research Challenges GPU Utilization as Measure of Inference Infrastructure

      • Deutschland - Deutsch
      • USA - English
      WEKA: The Foundation for Enterprise AI

      News provided by

      WEKA

      16 Nov, 2026, 22:00 CST

      Share this article

      Share this article


      New NAND Research report underscores that busy GPUs aren't an accurate measure of productive output; introduces new Inference Economics Scorecard to measure inference infrastructure by the useful work it delivers, as applied in an eight-hour benchmark with Nebius on NVIDIA HGX B300 GPUs and WEKA NeuralMesh.

      CHICAGO, Nov. 16, 2026 /PRNewswire/ -- From Supercomputing 2026: WEKA, the AI data and memory infrastructure company, in collaboration with market intelligence firm NAND Research, today introduced the Inference Economics Scorecard, a new common framework for measuring inference infrastructure productivity and economic value by the useful work it delivers rather than by how busy it keeps GPUs. The scorecard defines nine measures, including tokens per watt, cost per inference, and context retention. It sets four testing rules, including runs of at least eight hours and counting only successful output. Already applied in a benchmark with Nebius, the AI cloud company, it comes as inference is claiming a growing share of AI infrastructure spending and buyers hit power, memory, and rack-space limits.

      Continue Reading
      The New Math of Inference
      The New Math of Inference

      Buyers Are Already Treating Inference Differently, New Research Finds
      New research from 451 Research by S&P Global shows organizations are already adapting, projecting that inference will grow from 34% of AI infrastructure spending in 2026 to 46% by 2030 and finding that 41% of organizations running inference buy fundamentally different infrastructure for it than for the rest of their IT stack.

      "Inference is where the value of AI is ultimately realized, and our research shows it's reshaping how organizations buy infrastructure," said Greg Macatee, Senior Analyst at 451 Research by S&P Global. "Agents are driving exponential growth in token volume; memory, power, and supply constraints greatly challenge the notion that organizations can keep up simply by adding more GPUs."

      Why GPU Utilization Falls Short for Inference

      For years, training was the dominant AI workload. Because training keeps GPUs busy with large, predictable jobs, GPU utilization became the default measure of AI infrastructure productivity. Inference, especially agentic inference, places very different demands on GPUs. An agent does not simply answer a prompt; it completes a task over many steps, and at each step it re-reads the context it built earlier. After the GPU evicts that context from memory to make room for other users and sessions, it must recompute it when the task returns, and the task makes no further progress until that work is done. The result is a GPU that is fully utilized but unproductive because it wastes cycles recomputing the same tokens multiple times.

      The Inference Economics Scorecard measures inference systems at production scale: completed requests across many concurrent users, sustained over hours rather than minutes.

      "The industry has been grading inference with a training-era report card. GPU utilization tells you the chip was busy. It cannot tell you whether the work was finished, repeated, or thrown away, and with agents, a lot of it is repeated," said Liran Zvibel, co-founder and CEO of WEKA. "Power, supply, and rack space are constraints now as much as capital. The new math required for inference infrastructure is useful work per watt, per rack, per dollar, at the response times your users expect. The Inference Economics Scorecard puts that math on paper, and WEKA will be measured by it like everyone else."

      From Busy to Productive: How the Scorecard Works
      The Inference Economics Scorecard defines nine measures of inference performance and economics: tokens per watt, tokens per rack, cost per inference, GPU goodput, time to first token and end-to-end latency, memory efficiency, operational density, agent concurrency, and context retention. It also sets the discipline for testing against them:

      • The same workload and service targets across every system tested
      • Measurement of the whole serving stack
      • Runs of at least eight hours
      • Only successful output counted

      "The most expensive mistake in AI infrastructure right now is buying for peak throughput and paying for repeated work," said Steve McDowell, principal analyst and founder at NAND Research. "Agentic workloads make that mistake easy to miss, because every step of a long task re-reads context that a busy GPU may have to rebuild from scratch. We designed the scorecard so buyers can hold any system to the same standard: finished work, measured over hours, with every failure counted. The Nebius benchmark shows what that discipline reveals, because it ran for the full eight hours and counted every request that failed or timed out."

      Putting It to the Test: Real Agent Load, Sustained for Eight Hours
      WEKA and Nebius recently applied the scorecard's rules in a joint benchmark on NVIDIA HGX B300 systems and reported results on several measures. The test ran on one eight-GPU node serving DeepSeek-V4-Pro to 512 active sessions drawn from a corpus of more than 5,000 real agentic coding sessions over eight hours per configuration, counting every failed and timed-out request.

      Using WEKA NeuralMesh with Augmented Memory Grid, which pools NVMe across the cluster as a shared context tier, the node produced 3.7x the output tokens per GPU-hour compared with HBM-only serving on the same GPUs, and 97% of successfully completed requests reached a first token within five seconds, up from 44%. On HBM-only serving, 60% of prefill work was recomputed from context the cluster had already processed earlier in the session; with Augmented Memory Grid, 6.8%. At the measured rate, the HBM-only node would need about 30 hours to produce what Augmented Memory Grid produced in eight. The full benchmark is published at weka.io and nebius.com blogs.

      "Peak throughput shows what a system can do for a few minutes, but our customers run agents for hours, days, or longer, so what matters is sustained useful work," said Narek Tatevosyan, Vice President of Product at Nebius. "Over a full shift, the HBM-only configuration steadily lost ground as context fell out of memory and timeouts climbed, while WEKA NeuralMesh kept first-token latency flat the entire run. That changes how we plan capacity: we size for finished work over a working day, not a benchmark peak, and the Inference Economics Scorecard gives us a common way to show customers that number."

      "Agentic inference is the workload that defines the AI factory, and it puts extraordinary demand on memory and context. The question at scale is no longer how much context fits on the GPU, but how far it can be extended across the cluster without being recomputed," said Jason Hardy, VP, Storage Technology at NVIDIA. "The WEKA and Nebius results on NVIDIA HGX B300 systems show what a shared context tier can do for sustained throughput and responsiveness, and as inference grows, buyers will need a clearer way to compare systems on the work they actually finish."

      Reproducible by Design: Open Traces and Tooling
      The traces and tooling behind the benchmark are open-sourced so anyone can run the same workload on their own systems and compare results on equal terms. Because the traces come from real agentic coding sessions rather than synthetic load, and include the varied timing and complexity of real work, they give any team a realistic stand-in for production. Others are already building on them: SemiAnalysis's AgentX benchmark is built on the traces, and NVIDIA AIPerf benchmarking tool ships a loader for them.

      "A benchmark for agentic inference must reflect real-world production traffic, including long-running agents, growing context, complex multi-agent workflows with subagents, and the bursty, unpredictable traffic patterns that characterize real-world deployments," said Dylan Patel, founder and CEO at SemiAnalysis. "Too few benchmarks meet that bar, which is the gap we built AgentX to close, and it's why AgentX runs on real-world agentic traces using WEKA's open kvlab trace format, so anyone can replay the same multi-turn workflows and compare results on equal terms. As agents become the dominant inference workload, reproducible measurement like this is how the industry gets past peak numbers and sees what its infrastructure can sustain."

      SemiAnalysis plans to share AgentX results on WEKA NeuralMesh as testing progresses. For more information, visit https://inferencex.semianalysis.com/agentx.

      Additional Resources

      • The Inference Economics Scorecard and NAND Research companion report are available HERE.
      • Download a complimentary copy of the 451 Research by S&P Global report, "From inference to agentic AI" HERE.
      • Read the blog post from WEKA CEO Liran Zvibel. 
      • To learn about the engineering behind the capture tooling and how the measurements were defined, visit HERE.
      • Visit WEKA at SC26 in Chicago, November 16 to 20, booth #2519 to learn more about how to apply the Inference Economics Scorecard to your own inference workloads.

      About WEKA
      WEKA is the AI data and memory infrastructure company transforming the economics of agentic AI. The WEKA NeuralMesh platform unifies high-performance data storage with extended GPU memory, giving enterprises, AI cloud providers, and AI builders a single foundation for training, inference, and agentic workloads. With Augmented Memory Grid, NeuralMesh extends GPU memory capacity by 1000x, accelerates time to first token by up to 20x, and delivers 10x more concurrent users from the same GPU footprint, as proven in production benchmarks. WEKA enables organizations to scale production AI faster, get more useful work from every GPU, and reduce the cost of every token served. Learn more at www.weka.io or connect with us on LinkedIn and X.

      WEKA and the WEKA logo are registered trademarks, and WEKA NeuralMesh and Augmented Memory Grid are trademarks of WekaIO, Inc. Other trademarks mentioned herein are the property of their respective owners.

      SOURCE WEKA

      Modal title

      Also from this source

      WEKA and Andromeda Partner to Power AI Workloads at Global Scale

      WEKA and Andromeda Partner to Power AI Workloads at Global Scale

      WEKA, the AI data and memory infrastructure company, today announced that Andromeda, the platform connecting AI teams with high-performance compute,...

      WEKA Unveils WEKApod 3: The World's Densest AI Storage and Memory System for Agentic Workloads

      WEKA Unveils WEKApod 3: The World's Densest AI Storage and Memory System for Agentic Workloads

      WEKA, the AI data and memory infrastructure company, today unveiled the third generation of its WEKApod Nitro, WEKApod Prime, and WEKApod Prime Max...

      More Releases From This Source

      Explore

      Broadcast Tech

      Broadcast Tech

      Computer & Electronics

      Computer & Electronics

      Computer Hardware

      Computer Hardware

      Data Analytics

      Data Analytics

      News Releases in Similar Topics

      Contact PR Newswire

      • General Inquiries
      • Request a Demo
      • Partnerships
      • Media Inquiries

      Products

      • Explore Our Platform
      • Plan Campaigns
      • Create with AI
      • Distribute Press Releases
      • Report Results
      • Amplify Content

      About

      • About PR Newswire
      • About Cision
      • Become a Publishing Partner
      • Careers
      • Accessibility Statement
      • APAC – Simplified Chinese
      • APAC
      • APAC - Traditional Chinese
      • Arabic
      • Brazil
      • Canada
      • Czech
      • Denmark
      • Finland
      • France
      • Germany
      • India
      • Indonesia
      • Israel
      • Italy
      • Japan
      • Korea
      • Mexico
      • Middle East
      • Netherlands
      • Norway
      • Poland
      • Portugal
      • Russia
      • Slovakia
      • Spain
      • Sweden
      • United Kingdom
      • United States
      • Vietnam

      My Services

      • All New Releases
      • Platform

      Do not sell or share my personal information:

      • Submit via [email protected] 
      • Call Privacy toll-free: 877-297-8921

      Contact Cision

      Products

      About

      My Services
      • All News Releases
      • Platform
      [email protected]
      • Terms of Use
      • Privacy Policy
      • Information Security Policy
      • Site Map
      • RSS
      • Cookie Settings
      • Accessibility
      Copyright © 2026 Cision US Inc.