deepset Tackles AI costs with Agent Optimization and Custom Models for Government, Defense and Industry

New Haystack Enterprise Platform capabilities and engineering services help organizations run high-quality, sovereign AI agents more efficiently.

BERLIN, Oct. 21, 2026 /PRNewswire/ -- deepset, the German company behind the open source Haystack framework, today announced new capabilities and services to reduce the cost of running high-quality, sovereign AI agents. Unveiled during NVIDIA GTC Berlin, the offering combines agent optimization in Haystack Enterprise Platform with custom AI model services for government, defense and industry.

The announcement builds on deepset's work with organizations including the European Commission, whose self-hosted AI@EC platform supports governed AI applications; Germany's Federal Ministry of Research, Technology and Space (BMFTR), which uses an AI assistant for project-funding guidance; and Bosch, whose DRIVE Companion supports its SAP migration teams.

As AI usage grows, controlling AI spend requires looking beyond individual models to how they're used within agent systems. deepset helps organizations optimize model choice and the harness together, testing changes against cost and quality requirements while retaining control over models, data and infrastructure.

"AI economics will not be solved by choosing the cheapest model or defaulting to whichever model leads the market this week," said Milos Rusic, CEO of deepset. "The goal is to complete the task at the lowest cost that meets your quality and security requirements. Sometimes that means a smaller, specialized model. Sometimes it means improving the harness around a larger one. We help organizations work on both, measure the result and keep sovereign control of the system."

Reduce cost per task, maintain required quality

Haystack Enterprise Platform now shows AI spend by model, agent, service and team, both in centralized reporting and alongside production traces. Teams can find expensive or unreliable steps, compare models and harness configurations, and evaluate systematically whether a change reduces cost while meeting the quality required for the task.

As the platform covers the full loop of building, running, optimizing and operating agents, teams can identify and apply improvements rapidly within the same environment. Upcoming enhancements will further automate the loop, continuously recommending opportunities to reduce cost and improve quality of production agents, with human approval before deployment.

Use specialized models where they improve the economics

Industry-tuned decision models and small language models can handle repeated tasks such as classification, routing and verification more efficiently, reducing reliance on frontier LLMs, lowering costs and enabling deployment on locally controlled infrastructure.

deepset's custom AI model services bring its experience developing specialized language models to government, defense, energy and regulated industries. Forward Deployed Engineers assess customer requirements, select, adapt or post-train models, and evaluate them against quality and cost objectives before integrating them into production agent workflows.

Customers own the models developed for them. Haystack's model-agnostic orchestration connects those models with larger LLMs, enterprise data and tools, preserving the freedom to change components as new innovations or requirements emerge.

Efficiency with sovereign control

Organizations can deploy Haystack Enterprise Platform in public cloud, sovereign cloud environments such as STACKIT, or self-hosted infrastructure. Shared permissions, policies and usage controls govern how agents access models, tools and data.

This gives teams the ability to improve AI economics within their security and regulatory requirements, without tying their applications to a single model provider or deployment environment.

deepset and NVIDIA at GTC Berlin

Haystack Enterprise Platform uses NVIDIA Triton Inference Server for shared, GPU-accelerated embedding and reranking. Through its custom model services, deepset engineers can also assess and tailor NVIDIA Nemotron models where suited to customer tasks.

At GTC Berlin, deepset will join NVIDIA for sessions on Trustworthy AI Platforms and Secure, Self-Improving Agents for On-Prem AI. Visit deepset at booth #3071.

About deepset

Based in Berlin, deepset is the company behind Haystack, the widely adopted open source framework and enterprise platform for AI agent orchestration. Through its products and services, deepset helps organizations build, run, improve and govern sovereign AI agents while retaining control over models, data and infrastructure.  Learn more at: www.deepset.ai/

Media contact:
Jay Wilder
[email protected]
+1 732-737-1978

SOURCE deepset