deepset Unveils Reference Architecture for Self-Improving, Sovereign AI Agents on NVIDIA Infrastructure

New reference architecture combines open models, modular agent orchestration, and governed execution on infrastructure organizations control.

BERLIN, Oct. 21, 2026 /PRNewswire/ -- deepset, the company behind the open source Haystack AI orchestration framework and Haystack Enterprise Platform, today unveiled a new open reference architecture developed with NVIDIA open models and software for self-improving AI agents designed to run in locally controlled and air-gapped environments. The architecture is designed for organizations that need AI systems to adapt continuously while keeping sensitive data and workloads under their control.

Developed as part of an ongoing technical collaboration between the two companies, the architecture combines open models and NVIDIA technologies for model serving, optimization and governed execution with Haystack's modular agent orchestration.

Cybersecurity is the first proving ground. At NVIDIA GTC Berlin, the deepset will demonstrate the architecture through a cybersecurity agent that analyzes confidential source code for vulnerabilities and uses feedback from prior runs to refine its performance over time.

From "Deploy and Forget" to Continuous Improvement

At the center of the architecture are two Haystack agents that form a continuous feedback loop. The first performs the domain task. In the GTC demonstration, the agent analyzes confidential source code and vulnerability information within a locally controlled environment, using open models rather than sending sensitive code to external APIs.

A second Haystack agent analyzes traces and results from those runs and proposes changes to the agent harness. Each proposed change is evaluated against predefined objectives and the current version before it can be promoted, with approval, audit and rollback controls keeping people in charge of what reaches production.

The architecture combines four building blocks: open models for reasoning, a modular orchestration layer that connects those models to an organization's data and tools, an optimization environment where improvements can be evaluated, and security controls governing what agents are allowed to do.

Haystack provides the open source orchestration layer and agent harness, managing how models interact with context, retrieval, tools and workflows and making those components independently adaptable. running.The solution uses NVIDIA Nemotron open models served through NVIDIA NIM microservices, NVIDIA NeMo Gym as the optimization environment, and NVIDIA NemoClaw open blueprints, which run the agent inside NVIDIA OpenShell — a secure runtime that enforces policy-based controls over files, networks, credentials, and tools.  Nemotron 3 is a family of high-efficiency, open multimodal models with leading accuracy to deliver fastest time to completion for long-running multi-agent systems.

"Powerful open models create the foundation for sovereign AI, but realizing their full potential requires engineering the complete system around them," said Milos Rusic, CEO and co-founder of deepset. "By combining NVIDIA's open models and optimization technologies with Haystack's agent orchestration, organizations in Europe and beyond can build specialized agents that evolve with their environment while operating within governed infrastructure they control, and under human oversight."

"Organizations need agents that work effectively while retaining control over their data, infrastructure and operations," said Pat Lee, vice president of strategic partnerships at NVIDIA. deepset is providing a reference architecture organizations can deploy themselves, with the controls regulated industries require."

Sovereign by Architecture

For governments and regulated organizations in Europe and elsewhere, sovereignty requires more than running a model locally. Organizations also need control over what data and context an agent can access, which models and tools it uses, and the infrastructure on which the complete system operates.

The reference architecture is designed to preserve that control across the stack. Open models can run on locally controlled NVIDIA-accelerated infrastructure, including NVIDIA DGX Spark; Haystack keeps the agent harness modular across models, tools, and data sources; and OpenShell provides isolation and policy controls around agent execution.

Haystack's modular architecture allows organizations to adapt individual parts of the system as their requirements change, reducing dependence on a single model provider or closed application stack.

Explore the Architecture at NVIDIA GTC Berlin

deepset and NVIDIA will present "Evolve Within Bounds: Secure, Self-Improving Agents for On-Prem AI [SB61130]" at NVIDIA GTC Berlin on Thursday, October 22 at 10:00am CEST. Visit deepset at onsite at GTC at booth #3071.

A companion repository, available on deepset's GitHub page, provides example code to help developers explore and adapt the reference architecture. Developers can explore a related implementation in "Self-Improving AI Systems: A Log-Analysis Agent on NVIDIA DGX Spark," a technical walkthrough showing how a Haystack agent uses local models to analyze traces, evaluate configuration changes, and recommend improvements.

About deepset

Based in Berlin, deepset is the company behind Haystack, the widely adopted open source framework for AI agent orchestration. Through Haystack and Haystack Enterprise Platform, deepset provides the operating system for sovereign AI agents, helping organizations build, run, improve, and govern AI agents while retaining control over models, data tools, and infrastructure. Learn more at deepset.ai and start building with Haystack. 

Media Contact:
Jay Wilder
[email protected]
+1 732-737-1978 

SOURCE deepset