Accessibility Statement Skip Navigation
  • Resources
  • Investor Relations
  • Journalists
  • Agencies
  • Client Login
  • Send a Release
Return to PR Newswire homepage
  • News
  • Products
  • Contact
Search
When typing in this field, a list of search results will appear and be automatically updated as you type.

Searching for your content...

No results found. Please change your search terms and try again.
  • News Releases view all

    • News in Focus
        • Browse News Releases

        • All News Releases
        • All Public Company
        • English-only
        • News Releases Overview

        • Multimedia Gallery

        • All Multimedia
        • All Photos
        • All Videos
        • Multimedia Gallery Overview

        • Trending Topics

        • All Trending Topics
    • Business & Money
        • Auto & Transportation

        • Aerospace, Defense
        • Air Freight
        • Airlines & Aviation
        • Automotive
        • Maritime & Shipbuilding
        • Railroads and Intermodal Transportation
        • Supply Chain/Logistics
        • Transportation, Trucking & Railroad
        • Travel
        • Trucking and Road Transportation
        • Auto & Transportation Overview

        • View All Auto & Transportation

        • Business Technology

        • Blockchain
        • Broadcast Tech
        • Computer & Electronics
        • Computer Hardware
        • Computer Software
        • Data Analytics
        • Electronic Commerce
        • Electronic Components
        • Electronic Design Automation
        • Financial Technology
        • High Tech Security
        • Internet Technology
        • Nanotechnology
        • Networks
        • Peripherals
        • Semiconductors
        • Business Technology Overview

        • View All Business Technology

        • Entertain­ment & Media

        • Advertising
        • Art
        • Books
        • Entertainment
        • Film and Motion Picture
        • Magazines
        • Music
        • Publishing & Information Services
        • Radio & Podcast
        • Television
        • Entertainment & Media Overview

        • View All Entertain­ment & Media

        • Financial Services & Investing

        • Accounting News & Issues
        • Acquisitions, Mergers and Takeovers
        • Banking & Financial Services
        • Bankruptcy
        • Bond & Stock Ratings
        • Conference Call Announcements
        • Contracts
        • Cryptocurrency
        • Dividends
        • Earnings
        • Earnings Forecasts & Projections
        • Financing Agreements
        • Insurance
        • Investments Opinions
        • Joint Ventures
        • Mutual Funds
        • Private Placement
        • Real Estate
        • Restructuring & Recapitalization
        • Sales Reports
        • Shareholder Activism
        • Shareholder Meetings
        • Stock Offering
        • Stock Split
        • Venture Capital
        • Financial Services & Investing Overview

        • View All Financial Services & Investing

        • General Business

        • Awards
        • Commercial Real Estate
        • Corporate Expansion
        • Earnings
        • Environmental, Social and Governance (ESG)
        • Human Resource & Workforce Management
        • Licensing
        • New Products & Services
        • Obituaries
        • Outsourcing Businesses
        • Overseas Real Estate (non-US)
        • Personnel Announcements
        • Real Estate Transactions
        • Residential Real Estate
        • Small Business Services
        • Socially Responsible Investing
        • Surveys, Polls and Research
        • Trade Show News
        • General Business Overview

        • View All General Business

    • Science & Tech
        • Consumer Technology

        • Artificial Intelligence
        • Blockchain
        • Cloud Computing/Internet of Things
        • Computer Electronics
        • Computer Hardware
        • Computer Software
        • Consumer Electronics
        • Cryptocurrency
        • Data Analytics
        • Electronic Commerce
        • Electronic Gaming
        • Financial Technology
        • Mobile Entertainment
        • Multimedia & Internet
        • Peripherals
        • Social Media
        • STEM (Science, Tech, Engineering, Math)
        • Supply Chain/Logistics
        • Wireless Communications
        • Consumer Technology Overview

        • View All Consumer Technology

        • Energy & Natural Resources

        • Alternative Energies
        • Chemical
        • Electrical Utilities
        • Gas
        • General Manufacturing
        • Mining
        • Mining & Metals
        • Oil & Energy
        • Oil and Gas Discoveries
        • Utilities
        • Water Utilities
        • Energy & Natural Resources Overview

        • View All Energy & Natural Resources

        • Environ­ment

        • Conservation & Recycling
        • Environmental Issues
        • Environmental Policy
        • Environmental Products & Services
        • Green Technology
        • Natural Disasters
        • Environment Overview

        • View All Environ­ment

        • Heavy Industry & Manufacturing

        • Aerospace & Defense
        • Agriculture
        • Chemical
        • Construction & Building
        • General Manufacturing
        • HVAC (Heating, Ventilation and Air-Conditioning)
        • Machinery
        • Machine Tools, Metalworking and Metallurgy
        • Mining
        • Mining & Metals
        • Paper, Forest Products & Containers
        • Precious Metals
        • Textiles
        • Tobacco
        • Heavy Industry & Manufacturing Overview

        • View All Heavy Industry & Manufacturing

        • Telecomm­unications

        • Carriers and Services
        • Mobile Entertainment
        • Networks
        • Peripherals
        • Telecommunications Equipment
        • Telecommunications Industry
        • VoIP (Voice over Internet Protocol)
        • Wireless Communications
        • Telecommunications Overview

        • View All Telecomm­unications

    • Lifestyle & Health
        • Consumer Products & Retail

        • Animals & Pets
        • Beers, Wines and Spirits
        • Beverages
        • Bridal Services
        • Cannabis
        • Cosmetics and Personal Care
        • Fashion
        • Food & Beverages
        • Furniture and Furnishings
        • Home Improvement
        • Household, Consumer & Cosmetics
        • Household Products
        • Jewelry
        • Non-Alcoholic Beverages
        • Office Products
        • Organic Food
        • Product Recalls
        • Restaurants
        • Retail
        • Supermarkets
        • Toys
        • Consumer Products & Retail Overview

        • View All Consumer Products & Retail

        • Entertain­ment & Media

        • Advertising
        • Art
        • Books
        • Entertainment
        • Film and Motion Picture
        • Magazines
        • Music
        • Publishing & Information Services
        • Radio & Podcast
        • Television
        • Entertainment & Media Overview

        • View All Entertain­ment & Media

        • Health

        • Biometrics
        • Biotechnology
        • Clinical Trials & Medical Discoveries
        • Dentistry
        • FDA Approval
        • Fitness/Wellness
        • Health Care & Hospitals
        • Health Insurance
        • Infection Control
        • International Medical Approval
        • Medical Equipment
        • Medical Pharmaceuticals
        • Mental Health
        • Pharmaceuticals
        • Supplementary Medicine
        • Health Overview

        • View All Health

        • Sports

        • General Sports
        • Outdoors, Camping & Hiking
        • Sporting Events
        • Sports Equipment & Accessories
        • Sports Overview

        • View All Sports

        • Travel

        • Amusement Parks and Tourist Attractions
        • Gambling & Casinos
        • Hotels and Resorts
        • Leisure & Tourism
        • Outdoors, Camping & Hiking
        • Passenger Aviation
        • Travel Industry
        • Travel Overview

        • View All Travel

    • Policy & Public Interest
        • Policy & Public Interest

        • Advocacy Group Opinion
        • Animal Welfare
        • Congressional & Presidential Campaigns
        • Corporate Social Responsibility
        • Domestic Policy
        • Economic News, Trends, Analysis
        • Education
        • Environmental
        • European Government
        • FDA Approval
        • Federal and State Legislation
        • Federal Executive Branch & Agency
        • Foreign Policy & International Affairs
        • Homeland Security
        • Labor & Union
        • Legal Issues
        • Natural Disasters
        • Not For Profit
        • Patent Law
        • Public Safety
        • Trade Policy
        • U.S. State Policy
        • Policy & Public Interest Overview

        • View All Policy & Public Interest

    • People & Culture
        • People & Culture

        • Aboriginal, First Nations & Native American
        • African American
        • Asian American
        • Children
        • Diversity, Equity & Inclusion
        • Hispanic
        • Lesbian, Gay & Bisexual
        • Men's Interest
        • People with Disabilities
        • Religion
        • Senior Citizens
        • Veterans
        • Women
        • People & Culture Overview

        • View All People & Culture

    • Explore Our Platform
    • Plan Campaigns
    • Create with AI
    • Distribute Press Releases
    • Report Results
    • Amplify Content
    • All Products
    • General Inquiries
    • Editorial Bureaus
    • Partnerships
    • Media Inquiries
    • Worldwide Offices
    • Hamburger menu
    • PR Newswire: news distribution, targeting and monitoring
    • Send a Release
      • ALL CONTACT INFO
      • Contact Us

        888-776-0942
        from 8 AM - 10 PM ET

    • Searching for your content...

      No results found. Please change your search terms and try again.

      News Releases

      view all
      Close menu
    • News
    • Products
    • Contact
      • Send a Release
      • Client Login
      • Resources
      • Blog
      • Journalists
      • RSS
      • 3 News in Focus
      • 5 Business & Money
      • 5 Science & Tech
      • 5 Lifestyle & Health
      • 1 Policy & Public Interest
      • 1 People & Culture
      • Send a Release
      • Client Login
      • Resources
      • Blog
      • Journalists
      • RSS
      • Explore Our Platform
      • Plan Campaigns
      • Create with AI
      • Distribute Press Releases
      • Report Results
      • Amplify Content
      • All Products
      • Send a Release
      • Client Login
      • Resources
      • Blog
      • Journalists
      • RSS
      • General Inquiries
      • Editorial Bureaus
      • Partnerships
      • Media Inquiries
      • Worldwide Offices
      • Send a Release
      • Client Login
      • Resources
      • Blog
      • Journalists
      • RSS
      Close submenu (News in Focus) News in Focus
      • Browse News Releases
      • Multimedia Gallery
      • Trending Topics
      Close submenu (Business & Money) Business & Money
      • Auto & Transportation
      • Business Technology
      • Entertain­ment & Media
      • Financial Services & Investing
      • General Business
      Close submenu (Science & Tech) Science & Tech
      • Consumer Technology
      • Energy & Natural Resources
      • Environ­ment
      • Heavy Industry & Manufacturing
      • Telecomm­unications
      Close submenu (Lifestyle & Health) Lifestyle & Health
      • Consumer Products & Retail
      • Entertain­ment & Media
      • Health
      • Sports
      • Travel
      Close submenu (Policy & Public Interest) Policy & Public Interest
      • Policy & Public Interest
      Close submenu (People & Culture) People & Culture
      • People & Culture

      New Research Challenges GPU Utilization as Measure of Inference Infrastructure

      • APAC - English
      • Deutschland - Deutsch
      WEKA: The Foundation for Enterprise AI

      News provided by

      WEKA

      Nov 16, 2026, 09:00 ET

      Share this article

      Share this article


      New NAND Research report underscores that busy GPUs aren't an accurate measure of productive output; introduces new Inference Economics Scorecard to measure inference infrastructure by the useful work it delivers, as applied in an eight-hour benchmark with Nebius on NVIDIA HGX B300 GPUs and WEKA NeuralMesh.

      CHICAGO, Nov. 16, 2026 /PRNewswire/ -- From Supercomputing 2026: WEKA, the AI data and memory infrastructure company, in collaboration with market intelligence firm NAND Research, today introduced the Inference Economics Scorecard, a new common framework for measuring inference infrastructure productivity and economic value by the useful work it delivers rather than by how busy it keeps GPUs. The scorecard defines nine measures, including tokens per watt, cost per inference, and context retention. It sets four testing rules, including runs of at least eight hours and counting only successful output. Already applied in a benchmark with Nebius, the AI cloud company, it comes as inference is claiming a growing share of AI infrastructure spending and buyers hit power, memory, and rack-space limits.

      Continue Reading
      The New Math of Inference
      The New Math of Inference

      Buyers Are Already Treating Inference Differently, New Research Finds
      New research from 451 Research by S&P Global shows organizations are already adapting, projecting that inference will grow from 34% of AI infrastructure spending in 2026 to 46% by 2030 and finding that 41% of organizations running inference buy fundamentally different infrastructure for it than for the rest of their IT stack.

      "Inference is where the value of AI is ultimately realized, and our research shows it's reshaping how organizations buy infrastructure," said Greg Macatee, Senior Analyst at 451 Research by S&P Global. "Agents are driving exponential growth in token volume; memory, power, and supply constraints greatly challenge the notion that organizations can keep up simply by adding more GPUs."

      Why GPU Utilization Falls Short for Inference

      For years, training was the dominant AI workload. Because training keeps GPUs busy with large, predictable jobs, GPU utilization became the default measure of AI infrastructure productivity. Inference, especially agentic inference, places very different demands on GPUs. An agent does not simply answer a prompt; it completes a task over many steps, and at each step it re-reads the context it built earlier. After the GPU evicts that context from memory to make room for other users and sessions, it must recompute it when the task returns, and the task makes no further progress until that work is done. The result is a GPU that is fully utilized but unproductive because it wastes cycles recomputing the same tokens multiple times.

      The Inference Economics Scorecard measures inference systems at production scale: completed requests across many concurrent users, sustained over hours rather than minutes.

      "The industry has been grading inference with a training-era report card. GPU utilization tells you the chip was busy. It cannot tell you whether the work was finished, repeated, or thrown away, and with agents, a lot of it is repeated," said Liran Zvibel, co-founder and CEO of WEKA. "Power, supply, and rack space are constraints now as much as capital. The new math required for inference infrastructure is useful work per watt, per rack, per dollar, at the response times your users expect. The Inference Economics Scorecard puts that math on paper, and WEKA will be measured by it like everyone else."

      From Busy to Productive: How the Scorecard Works
      The Inference Economics Scorecard defines nine measures of inference performance and economics: tokens per watt, tokens per rack, cost per inference, GPU goodput, time to first token and end-to-end latency, memory efficiency, operational density, agent concurrency, and context retention. It also sets the discipline for testing against them:

      • The same workload and service targets across every system tested
      • Measurement of the whole serving stack
      • Runs of at least eight hours
      • Only successful output counted

      "The most expensive mistake in AI infrastructure right now is buying for peak throughput and paying for repeated work," said Steve McDowell, principal analyst and founder at NAND Research. "Agentic workloads make that mistake easy to miss, because every step of a long task re-reads context that a busy GPU may have to rebuild from scratch. We designed the scorecard so buyers can hold any system to the same standard: finished work, measured over hours, with every failure counted. The Nebius benchmark shows what that discipline reveals, because it ran for the full eight hours and counted every request that failed or timed out."

      Putting It to the Test: Real Agent Load, Sustained for Eight Hours
      WEKA and Nebius recently applied the scorecard's rules in a joint benchmark on NVIDIA HGX B300 systems and reported results on several measures. The test ran on one eight-GPU node serving DeepSeek-V4-Pro to 512 active sessions drawn from a corpus of more than 5,000 real agentic coding sessions over eight hours per configuration, counting every failed and timed-out request.

      Using WEKA NeuralMesh with Augmented Memory Grid, which pools NVMe across the cluster as a shared context tier, the node produced 3.7x the output tokens per GPU-hour compared with HBM-only serving on the same GPUs, and 97% of successfully completed requests reached a first token within five seconds, up from 44%. On HBM-only serving, 60% of prefill work was recomputed from context the cluster had already processed earlier in the session; with Augmented Memory Grid, 6.8%. At the measured rate, the HBM-only node would need about 30 hours to produce what Augmented Memory Grid produced in eight. The full benchmark is published at weka.io and nebius.com blogs.

      "Peak throughput shows what a system can do for a few minutes, but our customers run agents for hours, days, or longer, so what matters is sustained useful work," said Narek Tatevosyan, Vice President of Product at Nebius. "Over a full shift, the HBM-only configuration steadily lost ground as context fell out of memory and timeouts climbed, while WEKA NeuralMesh kept first-token latency flat the entire run. That changes how we plan capacity: we size for finished work over a working day, not a benchmark peak, and the Inference Economics Scorecard gives us a common way to show customers that number."

      "Agentic inference is the workload that defines the AI factory, and it puts extraordinary demand on memory and context. The question at scale is no longer how much context fits on the GPU, but how far it can be extended across the cluster without being recomputed," said Jason Hardy, VP, Storage Technology at NVIDIA. "The WEKA and Nebius results on NVIDIA HGX B300 systems show what a shared context tier can do for sustained throughput and responsiveness, and as inference grows, buyers will need a clearer way to compare systems on the work they actually finish."

      Reproducible by Design: Open Traces and Tooling
      The traces and tooling behind the benchmark are open-sourced so anyone can run the same workload on their own systems and compare results on equal terms. Because the traces come from real agentic coding sessions rather than synthetic load, and include the varied timing and complexity of real work, they give any team a realistic stand-in for production. Others are already building on them: SemiAnalysis's AgentX benchmark is built on the traces, and NVIDIA AIPerf benchmarking tool ships a loader for them.

      "A benchmark for agentic inference must reflect real-world production traffic, including long-running agents, growing context, complex multi-agent workflows with subagents, and the bursty, unpredictable traffic patterns that characterize real-world deployments," said Dylan Patel, founder and CEO at SemiAnalysis. "Too few benchmarks meet that bar, which is the gap we built AgentX to close, and it's why AgentX runs on real-world agentic traces using WEKA's open kvlab trace format, so anyone can replay the same multi-turn workflows and compare results on equal terms. As agents become the dominant inference workload, reproducible measurement like this is how the industry gets past peak numbers and sees what its infrastructure can sustain."

      SemiAnalysis plans to share AgentX results on WEKA NeuralMesh as testing progresses. For more information, visit https://inferencex.semianalysis.com/agentx.

      Additional Resources

      • The Inference Economics Scorecard and NAND Research companion report are available HERE.
      • Download a complimentary copy of the 451 Research by S&P Global report, "From inference to agentic AI" HERE.
      • Read the blog post from WEKA CEO Liran Zvibel. 
      • To learn about the engineering behind the capture tooling and how the measurements were defined, visit HERE.
      • Visit WEKA at SC26 in Chicago, November 16 to 20, booth #2519 to learn more about how to apply the Inference Economics Scorecard to your own inference workloads.

      About WEKA
      WEKA is the AI data and memory infrastructure company transforming the economics of agentic AI. The WEKA NeuralMesh platform unifies high-performance data storage with extended GPU memory, giving enterprises, AI cloud providers, and AI builders a single foundation for training, inference, and agentic workloads. With Augmented Memory Grid, NeuralMesh extends GPU memory capacity by 1000x, accelerates time to first token by up to 20x, and delivers 10x more concurrent users from the same GPU footprint, as proven in production benchmarks. WEKA enables organizations to scale production AI faster, get more useful work from every GPU, and reduce the cost of every token served. Learn more at www.weka.io or connect with us on LinkedIn and X.

      WEKA and the WEKA logo are registered trademarks, and WEKA NeuralMesh and Augmented Memory Grid are trademarks of WekaIO, Inc. Other trademarks mentioned herein are the property of their respective owners.

      SOURCE WEKA

      Modal title

      Also from this source

      WEKA Names Nic Morales as First Chief Customer Officer

      WEKA Names Nic Morales as First Chief Customer Officer

      WEKA, the AI data and memory infrastructure company, today announced the appointment of technology veteran Nic Morales as its first Chief Customer...

      WEKA and Andromeda Partner to Power AI Workloads at Global Scale

      WEKA and Andromeda Partner to Power AI Workloads at Global Scale

      WEKA, the AI data and memory infrastructure company, today announced that Andromeda, the platform connecting AI teams with high-performance compute,...

      More Releases From This Source

      Explore

      Broadcast Tech

      Broadcast Tech

      Computer & Electronics

      Computer & Electronics

      Computer Hardware

      Computer Hardware

      Data Analytics

      Data Analytics

      News Releases in Similar Topics

      Contact PR Newswire

      • Call PR Newswire at 888-776-0942
        from 8 AM - 9 PM ET
      • Chat with an Expert
      • General Inquiries
      • Editorial Bureaus
      • Partnerships
      • Media Inquiries
      • Worldwide Offices

      Products

      • Explore Our Platform
      • Plan Campaigns
      • Create with AI
      • Distribute Press Releases
      • Report Results
      • Amplify Content

      About

      • About PR Newswire
      • About Cision
      • Careers
      • Accessibility Statement
      • APAC
      • APAC - Simplified Chinese
      • APAC - Traditional Chinese
      • Brazil
      • Canada
      • Czech
      • Denmark
      • Finland
      • France
      • Germany
      • India
      • Indonesia
      • Israel
      • Italy
      • Japan
      • Korea
      • Mexico
      • Middle East
      • Middle East - Arabic
      • Netherlands
      • Norway
      • Poland
      • Portugal
      • Russia
      • Slovakia
      • Spain
      • Sweden
      • United Kingdom
      • Vietnam

      My Services

      • All New Releases
      • Platform Login
      • Data Privacy

      Do not sell or share my personal information:

      • Submit via [email protected] 
      • Call Privacy toll-free: 877-297-8921

      Contact PR Newswire

      Products

      About

      My Services
      • All News Releases
      • Platform Login
      • ProfNet
      Call PR Newswire at
      888-776-0942
      • Terms of Use
      • Privacy Policy
      • Information Security Policy
      • Site Map
      • RSS
      • Cookies
      Copyright © 2026 Cision US Inc.